I see many comments here taking an “inside view”, discussing Musk himself. Let’s take an “outside view” and look at the top 3 billionaires over the past 10 years, and see how long they stayed at that position. Note, this is based on Wikipedia’s [The World’s Billionaires](https://en.wikipedia.org/wiki/The_World%27s_Billionaires), which is based on the Forbes list, not the Bloomberg one. Rank 1: * Jeff Bezos: 2021, 2020, 2019, 2018 (4 consecutive years) * Bill Gates: 2017, 2016, 2015, 2014 (4 consecutive years) * Carlos Slim: 2013, 2012, 2011 (3 consecut...

Since mid March 2022, the community prediction on [Date Weakly General AI is Publicly Known](https://www.metaculus.com/questions/3479/date-weakly-general-ai-system-is-devised/) went down by 18 years, and [Date of Artificial General Intelligence](https://www.metaculus.com/questions/5121/date-of-general-ai/) went down by 23 years. In particular, both dropped from a date after 2040 to a date before 2040. The community prediction on the latter now assigns a 60% probability that it will happen before 2040. The criteria for the question that this meta-questio...

[Riffusion](https://www.riffusion.com/about) is Stable Diffusion (the image generation model) fine-tuned on spectrograms. It does a decent job at generating music from a text prompt. Now, neural nets generating music is nothing new, see e.g. [Wavenet from 2016](https://www.deepmind.com/blog/wavenet-a-generative-model-for-raw-audio), or [Jukebox from 2020](https://openai.com/blog/jukebox/). We’ve also seen already that [the transformer architecture can be adapted to generate images instead of text](https://openai.com/blog/image-gpt/). But what surprises m...

Tim Beiko announced a preliminary total terminal difficulty of 58750000000000000000000. When the network will reach that depends on the hash rate, but it’s likely early September.

Yandex open-sourced YaLM-100B today, licensed Apache 2.0. The repository says:

The model is published under the Apache 2.0 license that permits both research and commercial use, Megatron-LM is licensed under the Megatron-LM license.

From a quick glance, YaLM-100B appears to be an instance of Megatron-LM. (So the network architecture is Megatron-LM, and the weights are provided by Yandex.)

@(casens) > Is there a way to confirm the number of parameters from the open source code? It is hard to tell from the code, but we can tell from the data. I downloaded the weights and wrote the following script to count them: ``` #!/usr/bin/env python3 import numpy as np import os import torch import zipfile import pickle total_params = 0 weights_dir = "yalm100b_checkpoint/weights" for fname in os.listdir(weights_dir): print(f"\n{fname}:") # The next few lines basically reimplement `torch.load`, # but without actually loading the tensor...

Microsoft just trained a 530 billion parameter model, “Megatron-Turing NLG 530B”.

Some reference class tennis: * Out of 106 participants who participated twice, only one participant won twice (Johnny Logan) ⇒ 1/66 ≈ 0.9% * Out of 66 past editions, only once did a participant win who won before (Johnny Logan) ⇒ 1/66 ≈ 1.5% * Out of 14 participants (up to 2020) who participated more than once and won at least once, only one participant won twice ⇒ 1/14 ≈ 7.1% * Out of 11 times (up to 2020) that a participant who won previously participated again, only once did one win a second time ⇒ 1/11 ≈ 9.1% * Out of the 61 times that Sweden partic...

Spoiler alert, this comment refers to the plot of No Time To Die.

At the end of every Bond movie, typically after the credits, it shows “James Bond will return”. With the events in No Time To Die, I kind of expected to see “007 will return” instead. It would at least have kept the options open. I stayed until after the credits, but it said “James Bond will return”.

Yesterday was the last Monday of 2021, and the Mantic Monday post on that day did not mention this question.

(Edit: I was wrong about this! Scott did mention the title, see above.)

— edited by oumeen

@optimaloption The Goerli merge was just completed!

The community prediction odds on the linked question went from 1.5 (60%) in December 2022 to 4 (80%) currently (February 2023). It would have to go to 82% this month to reach the tripling threshold.

[Lighthouse (one of the Ethereum consensus layer clients) made a new release that embeds the previously shared parameters)](https://github.com/sigp/lighthouse/releases/tag/v3.0.0). [Geth (one of the execution clients) did so as well.](https://github.com/ethereum/go-ethereum/releases/tag/v1.10.22) There could still be changes later, but as the currently agreed upon TTD is approaching, that becomes more and more risky (because node operators might miss the update, and the network could fork). There could be fluctuations in hash rate, but they would have to...

Maybe Scott will avoid mentioning this question, but discuss https://www.metaculus.com/questions/7976/acx-… instead, carefully worded to avoid mentioning this question itself.

OpenAI just announced Codex, a GPT model fine-tuned on code, with up to 12B parameters. https://arxiv.org/abs/2107.03374

Microsoft just trained a 530 billion parameter model, “Megatron-Turing NLG 530B”.

The community prediction on Date of Artificial General Intelligence now assigns a 60% probability of AGI before 2040, and I think that a system that would pass the resolution criteria of that question, would be able to pass the test in this question. I wrote a more elaborate comment on the meta-question about this question.

[OpenAI refers to ChatGPT as “GPT-3.5”.](https://beta.openai.com/docs/model-index-for-researchers). ChatGPT/InstructGPT is publicly accessible, and [the paper that describes its architecture is on ArXiv](https://arxiv.org/abs/2203.02155). What I don’t know though, is whether it was trained using 5× more compute than GPT-3. The number of parameters is not different, but the training data is. [With the new scaling laws](https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scaling-laws-for-large-language-models), I can imagine that keeping the number of pa...

https://paperswithcode.com/sota/language-mode… hasn’t been updated for almost three years, but there has been lots of progress on language models. I expect that if you’d run LaMDA or ChatGPT against the Penn Treebank, they would score better than GPT-3, it’s just that nobody reported the set perplexity yet.

There is a market about the same topic on Polymarket, which at the time of writing has a price of $0.93 for “no” (vs $0.07 for “yes”) to “Will Ethereum switch to Proof-of-Stake (EIP-3675) by February 22, 2022?”