Rendered at 20:28:22 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
throwaway219450 33 minutes ago [-]
Astra writes well (which is why I often like to work with it over Claude), but it tends to produce code golf that's borderline unreadable, like completely omitting line breaks. I thought I had something funky with my config, but it seems to be a known issue.
The generated code certainly does what's asked of it, but it's as sloppy as anything else I've seen. I don't think this benchmark can control for that. Maybe if it had a metric for minimum diff to satisfy the new requirement with the existing constraints on dead code etc?
eptcyka 25 minutes ago [-]
Don’t apologize for x, just don’t post x links.
hedgehog 1 hours ago [-]
Is there a reasonable way to run a representative subset myself?
Edit:
> For folks who want to try to repro these findings: I used the pinned v1.0 catalog, the stock just-solve prompt, seed 42, the any-case runner pass policy, and Docker with Python 3.12 and uv.
For most things I find medium to be the highest quality output. However if I want it to do spatial reasoning want it be funny / write something (not code) of almost-passable quality, then xhigh is where it's at.
drivebyhooting 37 minutes ago [-]
First of all, what happened to Max and Ultra?
Secondly, I’m mostly using Astra Ultra or Sol 6.1 when tight on quota. I pay $200/month and rarely if ever run out. Am I doing something wrong? Or perhaps I’m not that productive as you guys?
steinvakt2 29 minutes ago [-]
Same here. I’m feeling productive enough. If I was more “productive”, I wouldn’t be able to keep up with what I was producing. My attention and cognitive broadband is the bottleneck. I find it hard to believe that people are actually productive using 4 active chat windows simultaneously for 8+ hours a day.
I believe effort can mean both "solve harder problems" and "produce more stuff". I mostly don't want more stuff than I asked for, so a capable model at lower effort can indeed be the wiser choice. Faster response too.
Onavo 1 hours ago [-]
I wish a company named human layer won't write their reports with so much Claudisms.
1 hours ago [-]
0c3ca83 1 hours ago [-]
You expect an AI company to slow down and care about what they produce? How naive are you?
If you haven't pushed so much that people are physically unable to look at it, are you even running a company?
Onavo 28 minutes ago [-]
I think people downvoting you missed the sarcasm.
perching_aix 6 minutes ago [-]
On the contrary.
cute_boi 58 minutes ago [-]
The irony is the article is very sloppy. Can we at least write it and review it properly..
antipaul 7 minutes ago [-]
This is true. I was unfamiliar with this concept (I use codex/claude daily), and am trying to get to the "bottom line".
Oh, it's there at bottom under "my takeaway". But I still didn't get it initially.
https://lucumr.pocoo.org/2026/9/7/astra-why/
[Sorry for X] https://x.com/tenobrus/status/2096636223414808719
The generated code certainly does what's asked of it, but it's as sloppy as anything else I've seen. I don't think this benchmark can control for that. Maybe if it had a metric for minimum diff to satisfy the new requirement with the existing constraints on dead code etc?
Edit:
> For folks who want to try to repro these findings: I used the pinned v1.0 catalog, the stock just-solve prompt, seed 42, the any-case runner pass policy, and Docker with Python 3.12 and uv.
And https://github.com/gabeorlanski/scb-problems/
Secondly, I’m mostly using Astra Ultra or Sol 6.1 when tight on quota. I pay $200/month and rarely if ever run out. Am I doing something wrong? Or perhaps I’m not that productive as you guys?
Benchmarking Opus 5 on SlopCodeBench - https://news.ycombinator.com/item?id=49076391 - July 2026 (119 comments)
I believe effort can mean both "solve harder problems" and "produce more stuff". I mostly don't want more stuff than I asked for, so a capable model at lower effort can indeed be the wiser choice. Faster response too.
If you haven't pushed so much that people are physically unable to look at it, are you even running a company?
Oh, it's there at bottom under "my takeaway". But I still didn't get it initially.
Slop Code Bench? Need a Slop Writing Bench too