LessWrong - A 90s GeoCities Tribute

Community Events

Event	Date	Location
Join Us for the Memory Decoding Journal Club!	Tue Apr 8	Online
Australia's AI Crossroads: Election 2025 Town Hall	Wed Apr 9	Online
04/07/25 Monday Social	Tue Apr 8	Houston
Wollongong – ACX Meetups Everywhere Spring 2025	Tue Apr 8	Wollongong

Subscribe (RSS/Email) | LW the Album | About | FAQ

Latest Posts

Predictable updating about AI risk
Best of LessWrong 2023
We shouldn't expect to get a lot more worried about AI risk as capabilities increase, if we're thinking about it clearly now. Joe discusses why this happens anyway, and how to avoid it.
by Joe Carlsmith

Recent | Enriched | Recommended | Customize

451	Welcome to LessWrong! Ruby, Raemon, RobertM, habryka
652	METR: Measuring AI Ability to Complete Long TasksΩ Zach Stein-Perlman
882	Good Research Takes are Not Sufficient for Good Strategic TakesΩ Neel Nanda
274	AI 2027: What Superintelligence Looks Like Daniel Kokotajlo, Thomas Larsen, elifland, Scott Alexander, Jonas V, romeo
452	LessWrong has been acquired by EA habryka
143	VDT: a solution to decision theory L Rudolf L
441	Will Jesus Christ return in an election year? Eric Neyman
475	Why Have Sentence Lengths Decreased? Arjun Panickssery
331	How to Make Superbabies GeneSmith, kman
572	Policy for LLM Writing on LessWrong jimrandomh

732	Recent AI model progress feels mostly like bullshit lc
226	Tracing the Thoughts of a Large Language ModelΩ Adam Jermyn
113	The Lizardman and the Black Hat Bobcat Screwtape
150	A Bear Case: My Predictions Regarding AI Progress Thane Ruthenis

Advanced Sorting/Filtering

Quick Takes | Cancel | Submit

J Bostock
From Rethink Priorities: Now with the disclaimer that I do think that RP are doing good and important work and are one of the few organizations seriously thinking about animal welfare priorities...

Their epistemics led them to do a Monte Carlo simulation to determine if organisms are capable of suffering (and if so, how much) then got a value of 5 shrimp = 1 human and then not bat an eye at this number. Neither a physicalist nor a functionalist theory of consciousness can reasonably justify a number like this. Shrimp have 5 orders of magnitude fewer neurons than humans, so whether suffering is the result of a physical process or an information processing one, this implies that shrimp neurons do 4 orders of magnitude more of this process per second than human neurons. The authors get around this by refusing to stake themselves on any theory of consciousness. The overall structure of the RP welfare range report, does not cut to the truth, instead the core mental motion seems to be to engage with as many existing piece of work as possible; credence is doled out to different schools of thought and pieces of evidence in a way which seems more like appeasement, lip-service, or a "well these guys have done some work, who are we disrespect them by ignoring it" attitude. Removal of noise is one of the most important functions of meta-analysis, and it is largely absent. The result of this is an epistemology where the accuracy of a piece of work is a monotonically increasing function of the number of sources, theories, and lines of argument. Which is fine if your desired output is a very long Google doc, and a disclaimer to yourself (and, more cynically, your funders) that "No no, we did everything right, we reviewed all the evidence and took it all into account." but it's pretty bad if you want to actually be correct. I grow increasingly convinced that the epistemics of EA are not especially good, worsening, and already insufficient to work on the relatively low-stakes

Kongo Landwalker
DEFCON Chess The game requires 1d6. The match is divided into 6 stages, named DEFCON 6 to DEFCON 1. The game starts at DEFCON 6. Before a player moves he is required to throw a dice: if the value is 1 the game progresses to the next stage. DEFCON 6: players are constrained to their home half of the board. DEFCON 5: players now can move anywhere on the board. DEFCON 4: captures are unlocked. DEFCON 3: special moves (pawn double move, en passant, promotion, castling) are unlocked. DEFCON 2: attacking piece also dies on capture event. DEFCON 1: atomic chess variant rules apply.

Simcha
I miss the days where when someone asked me for my political opinion I could say something interesting, maybe even something theoretical or abstract. Nowadays my opinions are all "tariffs bad" "free trade good" "due process good" I'm involuntarily being NPC'd. Things that used to be obvious as fact are now 'opinions' that must therefore be Debated, leaving true opinions in the realm of farfetched theory best left to quiet personal conversations - certainly not to real political commentary.

Stephen Fowler
Thinking of trying the latest Gemini model? Be aware that it is almost impossible to disable the "Gemini in Docs" and "Gemini in Gmail" services once you have purchased a Google One AI Premium plan. Edit: Spent 20 minutes trying to track down a button to turn it off before reaching out to support. A support person from Google told me that as I'd purchased the plan there was literally no way to disable having Gemini in my inbox and docs. Even cancelling my subscription would keep the service going until the end of the current billing period. But despite what support told me, I resolved the issue. Account > Data and privacy > "Delete a Google service" and then I deleted my Google One account. No more Gemini in inbox and my account on the Gemini app seems to have reverted to a free user account. I imagine this "solution" won't be feasible if you use Google One for anything else (file storage).

NormanPerlmutter
Things are getting scary with the Trump regime. Rule of law is breaking down with regard to immigration enforcement and basic human rights are not being honored. I'm kind of dumbfounded because this is worse than I expected things to get. Do any of you LessWrongers have a sense of whether these stories are exaggerated or if they can be taken at face value? Deporting immigrants is nothing new, but I don't think previous administrations have committed these sorts of human rights violations and due process violations. Krome detention center in Miami -- overcrowded and possibly without access to sufficient drinking water https://www.miamiherald.com/news/local/immigration/article303485356.html https://www.yahoo.com/news/unpacking-claims-ice-holding-4k-220400460.html https://www.instagram.com/jaxxchismetalk/reel/DHjxaXzAddP/ https://www.instagram.com/catpowerofficial/reel/DHhsiMvJ8BT/people-are-dying-under-ice-detainment-in-miamiand-this-video-is-from-last-weekpl/ Canadian in the US legally to apply for a work visa detained 2 weeks by ICE without due process https://www.theguardian.com/us-news/2025/mar/19/canadian-detained-us-immigration-jasmine-mooney
Load More (5/53)

Popular Comments

METR: Measuring AI Ability to Complete Long Tasks
224 Zach Stein-Perlman Ω 751 9d
This is a linkpost forhttps://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/Summary:We propose measuring AI performance in terms of thelengthof tasks AI agents can complete. We show that this metric has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months. Extrapolating this trend predicts that, in under five years, we will see AI agents that can independently complete a large fraction of software tasks that currently take humans days or weeks.The length of tasks (measured by how long they take human professionals) that generalist frontier model agents can complete autonomously with 50% reliability has been doubling approximately every 7 months for the last 6 years. The shaded region represents 95% CI calculated by hierarchical bootstrap over task families, tasks, and task attempts.Full paper|Github repoWe think...(Continue Reading – 1283 more words)shash4223m10These results empirically resolve for me why scaling will continue to be economically rational despite logarithmic gains in (many) benchmark performance.
https://www.lesswrong.com/posts/dAYemKXz4JDFQk8QE/log-linear-scaling-is-worth-the-cost-due-to-gains-in-long

How to Make Superbabies
569 GeneSmith,kman 2mo
EDIT:Read a summary of this post on TwitterWorking in the field of genetics is a bizarre experience. No one seems to be interested in the most interesting applications of their research.We’ve spent the better part of the last two decades unravelling exactly how the human genome works and which specific letter changes in our DNA affect things like diabetes risk or college graduation rates. Our knowledge has advanced to the point where, if we had a safe and reliable means of modifying genes in embryos, we could literally create superbabies. Children that would live multiple decades longer than their non-engineered peers, have the raw intellectual horsepower to do Nobel prize worthy scientific research, and very rarely suffer from depression or other mental health disorders.The scientific establishment,...(read more)

Log-linear Scaling is Worth the Cost due to Gains in Long-Horizon Tasks
10 shash4229m
This post makes a simple point, so it will be short. I am happy to discuss more in the comments, and based on this write a longer post later. Much prior work(eg: [1])has shown that exponential data and compute is required for each unit improvement in accuracy. A popular argument this leads to:Scaling compute and data is economically not viable above a thresholdThis argument has a key issue. Log-linear gains in accuracy (or loss) are shown on "one-step" benchmarks such as direct QA. However, economic benefits primarily arise from tasks which require long thinking, output and action horizons. For this, I propose trackingk-step success rate, i.e. the probability that a k-step solution achieves its goal. Most benchmarks previously analyzed for log-linear scaling have at...(See More–217more words)

How much progress actually happens in theoretical physics?
30 ChristianKl 3d
I frequently hear people make the claim that progress in theoretically physics is stalled, partly because all the focus is on String theory and String theory doesn't seem to pan out into real advances.Believing it fits my existing biases, but I notice that I lack the physics understanding to really know whether or not there's progress. What do you think?

Show, not tell: GPT-4o is more opinionated in images than in text
78 Daniel Tan,eggsyntax 6d
Epistemic status: This should be considered an interim research note. Feedback is appreciated.IntroductionWe increasingly expect language models to be ‘omni-modal’, i.e. capable of flexibly switching between images, text, and other modalities in their inputs and outputs. In order to get a holistic picture of LLM behaviour,black-box LLM psychologyshould take into account these other modalities as well.In this project, we do some initial exploration of image generation as a modality for frontier model evaluations, using GPT-4o’s image generation API. GPT-4o is one of the first LLMs to produce images natively rather than creating a text prompt which is sent to a separate image model, outputting images and autoregressive token sequences (ie in the same way as text).We find that GPT-4o tends to respond in a consistent manner...(See More–857more words)

Cohabitive Games so Far
131 mako yass 2y
This is a linkpost forhttps://makopool.com/peacewagers.htmlA cohabitive game[1]is a partially cooperative, partially competitive multiplayer game that provides an anarchic dojo for development in applied cooperative bargaining, or negotiation.Applied cooperative bargaining isn't currently taught, despite being aninfrastructural literacyfor peace, trade, democracy or any other form of pluralism. We suffer for that. There are many good board games that come close to meeting the criteria of a cohabitive game today, but they all[2]miss in one way or another, forbidding sophisticated negotiation from being practiced.So, over the past couple of years, we've been gradually and irregularly designing and playtesting the first[2]cohabitive boardgame, which for now we can callDifference and PeacePeacewager 1, or P1. This article explains why we think this new genre is important, how it's been going, what we've learned,...(Continue Reading – 5462 more words)

Ram Potham's Shortform
Ram Potham 15d
Accelerating AI Safety research is critical for developing aligned systems, andtransformative AI is a powerful approachin doing so.However to create AI systems that accelerate AI safety requires benchmarks to find the best performing system. The problem is that benchmarks can beflawed and gamedfor the following core reasons:Benchmarks may not represent real world abilityBenchmark information can be leaked into AI model trainingHowever, we can have amoving benchmarkof reproducing the most recent technical alignment research using the methods and...(read more)

AI 2027: What Superintelligence Looks Like
458 Daniel Kokotajlo,Thomas Larsen,elifland,Scott Alexander,Jonas V,romeo 4d
This is a linkpost forhttps://ai-2027.com/In 2021 I wrote what became my most popular blog post:What 2026 Looks Like. I intended to keep writing predictions all the way to AGI and beyond, but chickened out and just published up till 2026.Well, it's finally time. I'm back, and this time I have a team with me:the AI Futures Project.We've written a concrete scenario of what we think the future of AI will look like.We are highly uncertain, of course, but we hope this story will rhyme with reality enough to help us all prepare for what's ahead.You really should goread it on the websiteinstead of here, it's much better. There's asliding dashboardthat updates the stats as you scroll through the scenario!But I've nevertheless copied the...(Continue Reading – 12069 more words)

janshi
They expand their contract with OpenBrain to set up an “Oversight Committee,” a joint management committee of company and government representatives, with several government employees included alongside company leadership.The White House considers replacing the CEO with someone they trust, but backs off after intense employee protests.This seemed relatively less likely to me in 2027 compared to 2023 given that a few paragraphs earlier it is described thatBut Agent-4 now exercises significant control over OpenBrain’s day-to-day operation.How many hu...(read more)

ryan_greenblatt
Notably, the trend in the last few years is that AI companies triple their revenue each year. So, the revenue tripling seems very plausible to me. As far as 50% algorithmic progress, this happens using Agent-1 (probably with somewhat better post training then the original version) in around April 2026 (1 year from now). I think the idea is that by this point, you have maybe a 8-16 hour horizon length on relatively well contained benchmark tasks which allows for a bunch of the coding work to be automated including misc experiment running. (Presumably the horizon length is somewhat shorter on much messier tasks, but maybe by only like 2-4x or less.) Note that this only speeds up overall AI progress by around 25% because AI R&D maybe only drives a bit more than half of progress (with the rest driven by scaling up compute). Personally, I think 50% seems somewhat high given the level of capability and the amount of integration time, but not totally crazy. (I think I'd guess more like 25%? I generally think the speed ups they quote are somewhat too bullish.) I think I disagree more with the estimated current speed up of 13% (see April 2025). I'd guess more like 5% right now. If I bought that you get 13% now, I think that would update me most of the way to 50% on the later milestone.

Thane Ruthenis
Hm, I admittedly only skimmed the Compute Forecast article, but I don't think there's much evidence for a trend like this? The "triples every year" statement seems to be extrapolated from two data points about OpenAI specifically ("We use OpenAI’s 2023 revenue of $1B and 2024 revenue around $4B to to piece together a short term trend that we expect to slow down gradually", plus maybe this). I guess you can draw a straight line through two points, and the idea of this trend following straight lines doesn't necessarily seem unconvincing a-priori... But is there more data? Yeah, I concur with all of that: some doubts about 50% in April 2026, some doubts about 13% today, but seems overall not implausible.

ryan_greenblatt
I think the best source for revenue growth is this post from epoch. I think we only have the last 2 years really, (so "last few years" is maybe overstating it), but we do have revenue projections and we have more than 1 data point per year.

WELCOME TO LESSWRONG !