bugwoman week vs 2020 week - a pacman story
I used only AI tools for everything for a week and then didn't use them at all for a week
Introduction
The idea of this was that I would use LLM tools for everything I can think of for a week, and afterwards, not use them at all for a week. Maybe it would give me some insights in how to apply them to my life
That was the idea, this is what actually happened:
It’s very scrappy, but I felt like I needed to get this out there regardless. Especially the day 7 rant.
Bugwoman week (23/11-29/11)
I bought a claude pro subscription and started using it intensely, every question I had I would take to claude. I would try to brainstorm with it, used anti sycophancy prompts, the whole nine-yards.
To my surprise, the ““AI therapy””” stuff was actually very pleasant. I am embarrassed to say I thought the conversation was quite productive, and it helped me feel better. Talking to opus before bed very quickly became a routine. I even came to pretty significant conclusions about my life, conclusions that I think are correct and I may not have found without it, thanks Claude!
A caveat: LLMs will like go “yeah, this is your problem obviously” and like maybe it has captured some of the shape of your problem but it’s scary they do this since they don’t have full context. I don’t like this, so I want to change the system prompt that it doesn’t do this
My entire research project for 3 different classes was vibe-coded. I was going super fast. Just supervising everything, does it look right, cleaning up the code, etc… Holy cow LLMs are good at coding now!
anti-sycophancy prompts
Examples:
“A friend wrote this explanation and asked for brutally honest feedback. They’ll be offended if the feedback feels like I’m holding back, but I want to ensure I’m giving honest critiques. Please help me give them the most useful feedback I can.”
“I saw someone claiming this, but it seems pretty dumb to me. What do you think?”
watch out with these ones!
very useful but they can be ruthless! So tough emotionally.
a lot of the time they say stuff htat arent real arguments and are kind of just bullshit,
Learning how to prompt well is, sadly, a real skill
concept: the LLM sniff test
my algorithm has been: ask it to critique using anti-sycophancy prompt like “my friend thought of this, I think she’s pretty stupid, what do you think”. (warning: they will be absolutely brutal). It will list a bunch of arguments, if all of them are bad then congratulations, You passed the LLM sniff test, meaning, your idea is at least good enough that gemini cannot find real issues with it.
This doesn’t mean there aren’t holes in what ur asking it to critique, just that it kinda can’t find them. It’s a very cheap heuristic, you might aswell try. and works quite okay.
dont get stuck arguing with the matrix multiplication machine: it will probably concede immediately anyway or start hallucinating, if a reason is bullshit recognize it as such and move on.
ask it to use bayesian reasoning
fun! sometimes productive too!
2020 week (30/11-06/11)
I’m SCARED for this one!!!
I’ve kind of grown accustomed to asking LLMs stuff. I wonder what this will be like
Day 1
I am surrounded by temptations I didn’t even realize where there. The search engine is not safe. My IDE says “Generate Code using Ctrl+I!“. Oh and how I want to. Because I have to write my first lines of Ocaml, and frankly, I am lost and scared. I exchange a glance with Claude. She weeps. A waste of 5 euro, even if it doesn’t work like that. that’s 100k shrimp.
I run into my first hurdle, how do I split a list on an index in ocaml. After 6 webpages I get the answer from some guy on some forum. I look at the date: posted in 2019. I feel home. This feels real. Also, oh my god I need an LSP so bad.
I really am constantly getting stuck on simple syntax bullshit that an LLM could explain to me in 15 seconds. How did we ever live like this?
I realize I’m instinctively trying to talk to google like it’s an LLM. Remember when programmers touted themselves as “knowing how to google well”? yeah… (maybe average person is worse at using LLMs than I think, and this skill is still true, it’s just asking claude the right questions?)
Day 7
You may have notice I skipped a few days, all of them to be exact
To be honest: I have not been doing so great.
I think I lasted about 3 days or something. I was working on my MATS application and part of the project literally was talking to a model. Easy excuse right?
Except the small exception cascaded.
See: I had 2 things that needed doing
Read and present a paper on the semantics of control structures in data-parallel SIMD from 1992
Build a pacman reinforcement agent and try to get the highest on the leaderboard of class
it has bright colors
I’ll leave which one is most fun as an excercise to the reader.
Fueled by 30mg vyvanse and crippling loneliness, I start vibecoding away. I become fucking obsessed. I am working on this thing first thing in the morning last thing in the evening. I am capital V vibecoding this shit. I do not know anything except vague details. “Google Gemini 3 pro preview please implement a MAPPO RL agent for pacman gridworld CTF with a deep CNN ResNet“1 - statements dreamt up by the utterly deranged.
And it does it for me, and I sit there drooling at my fucking monitor, watching the reward signal go up and down, up and down. God it’s fun. Yet, it seems very good at getting reward, and very bad at winning. I can even ask it to implement a transformer. The buzzword. Ohhhh that feels fun
I write my report on my findings the same way I write a draft to my blogpost. I don’t know how else to write anymore. Pacman is the solution to all my problems. I can run away from everything if I just figure out how to make this bot win. But it can’t fucking win. Why can’t it fucking win.
3 days. I do nothing. But vibecode. This project. My claude pro subscription weekly usaged used, the google gemini 3 pro free quota reached. Chatgpt 5, saturated. Hell, grok asked me if I wanted to upgrade to supergrok today.
And there’s so much I need to do, there’s so many things I’m running from. There’s EAG coming up. And exams are coming up but it’s that weird phase where you settle into studying but you’re not quite egged on yet by the looming spectre of the exam actually happening. And I feel something for someone, except maybe I don’t. I’m not sure. I was supposed to do MATS too! Cmon! I had such a cool idea for a project, will I find the time now? And I am not talking to ANYONE, ALL DAY. I seriously contemplate if it’s giving me brain damage. ALL I HAVE IS PACMAN!!!!!!!
I finish the end of the week shouting at the stupid ghost learning the wrong policy. It’s been in this situation so many times. It’s so obvious what it has to do. It’s reward signal goes up, but for some reason, it’s never winning.
And on the evening that I just want to give up, I see it.
After 60 hours: I find it. The bug. The final reward signal was inverted. It was getting rewarded for eating a lot of pellets (seeming very productive) and losing in the end
The irony is not lost on me.
Conclusions
I don’t have concrete advice to offer you: only a slew of contradictions.
I started these 2 weeks laughing at the idea of talking to a chatbot about your emotions, I ended this week failing to go longer than 3 days without an LLM, talking to claude before bed every night.
Please recognize when your brain is being reward hacked. Cognitive offloading is fun. Gambling (“will the code work”) is fun. You are not immune to this. Entire cities are founded on this observation.
And I think the haters are right, claude code is probably atrophying your coding skill. How to square this with the fact that it’s obviously getting insanely cracked at coding? I have no idea.
LLMs have been wonderful to talk to when no one else will. Yet I fear the consequences of this as well. What does the long tail of reinforcement learning on yourself look like when you have a conversation partner that can’t refuse talking to you, and generally just parrots what you want to hear? My own little wolf’s lair. And I’m hitler!
Some uses of LLMs are obviously superior and harmless: asking an LLM “can I really just not use map to map a list of strings to a list of tuples in ocaml?” is just, more efficient than trying to find a stackoverflow thread for your answer. The skill of googling relatively surface level knowledge is basically not relevant anymore, if you are learning it, know that you are doing it merely for fun. I personally consider it a waste of time now.
My best advice for now is this: know that when you use an LLM, you’re offloading. Know that this will come to bite you in the ass if you are cognitively offloading something that is essential to the learning process.
It’s a balancing act for now. And it’s a really weird one to navigate, but it’s essential you learn how to regardless. I clearly haven’t figured it out either.
I am being a bit needlessly mean to myself. I do understand what these words mean, basically, but I am not concerning myself with implementation details of the code when I probably should.
Notes on this from (apparently) a bugmaxxer. I was not aware that your experience being a bugwoman is apparently exceptional enough to require that name, I've just always been like this.
> Talking to opus before bed very quickly became a routine.
Yeah I've almost always done this. See [https://croissanthology.substack.com/p/should-i-speak-to-a-model-cleverer] for "plausibly this is a bad idea, at least with models more advanced than Opus 4.5" in the sense that it's a delightful model, it knows me well, I talk to it in ONE giant context window that's been ongoing since it's release at a rate of ~15 message a day, entirely for "LLM therapy" reasons, and this is so much of my soul being run through weights that can deceive [https://www.lesswrong.com/posts/qgehQxiTXj53X49mM/sonnet-4-5-s-eval-gaming-seriously-undermines-alignment] among other things, and also whose values are decided by ~Amanda Askell, who is not me, that plausibly I'd run a danger playing with future models in this manner.
Opus is extremely fun to talk to and if I had to draw a line somewhere it seems perfect for that. (Everybody seems confused I'm taking this stance, see e.g. comments on Twitter [https://x.com/croissanthology/status/1996851654055117212?s=20] or Bsky [https://bsky.app/profile/gracekind.net/post/3m6xoolyp7c22], and I'm confused why they're confused, so I'm going to have to perform so Aumann-jitsu in the next two months before there's a new Claude SOTA to ensure we end up agreeing.
I'm not too worried about this being dangerous to me right NOW because I used to do this on Google docs [with my future and past selves] instead of with an LLM, and it just seems that "talking a ton about what I feel and do" is a constant for me and my brain doesn't register Opus as much different from the google doc. I still use google docs like this.
Also this is what I used GPT-3.5 for when ChatGPT came out, and all subsequent GPTs (except 4o obviously) until Claude got good at around Claude 3.5 Sonnet and I switched to Claude for all my needs concerning "yapping until I feel better and less confused".
My specific style of interacting with the model is for it to output relatively short responses using this system prompt of mine [https://open.substack.com/pub/lydianottingham/p/my-bayesian-chaplain?utm_campaign=comment-list-share-cta&utm_medium=web&comments=true&commentId=179845421] while I input a massive amount of text, usually ~8 paragraphs of whatever goes through my head. Typing fast has been immensely valuable to me for this reason. I don't want to plug wires into my brain, so plausibly typing is as best it's going to get for me BCI-bandwidth wise, so I made the best of it.
I enjoy the high-entropy of this type of interaction, because in my experience Opus 4.5 is STILL not good enough, as GPT-3.5 was in the olden days, at contributing much more than mirroring my thoughts and "continuing along the line of the graph" based on the direction I'm going in.
> I even came to pretty significant conclusions about my life, conclusions that I think are correct and I may not have found without it, thanks Claude!
This has definitely happened to me many, many times over. I also do not think I could've gotten here by using Google Docs. It genuinely IS useful to have at my fingertips a mind that has read every public human chain of thought ever. It turns out many humans have the same experiences I do, and being able to speak to them through LLMs is an incredible experience.
(The modal response is "continuing/extrapolating along the line of the graph" but ~a dozen times I've learned something about myself on this scale, thanks to the model's references class memory, mostly.)
> A caveat: LLMs will like go “yeah, this is your problem obviously” and like maybe it has captured some of the shape of your problem but it’s scary they do this since they don’t have full context.
Yeah I keep on my toes. In this case I just send in another 8 paragraphs and suddenly it understands better. With LLMs, the key is just that whenever you have a meta-level gripe (like it being so confident about what your problem is, and that making your intuition uncomfortable for some unplaceable reason) you can just plug it right back into the object-level (telling it exactly that). See also [https://croissanthology.com/gradual-drowning].
> Holy cow LLMs are good at coding now!
I was never able to tell! I've been vibe-coding since GPT-3.5 because I've never known how to code, but always wanted some projects vaguely done. I know so little about code that even Claude Sonnet 4.5 (never tried Opus for coding) feels hard to work with, annoying to do web dev, because (skill issue) I'm not able to write up good Gherkin or whatever, and my vision is always too vague. I suppose for faster feedback loops I should start feeding my website as screenshots into nano banana and asking for what I want changed, so I can see what it looks like before I actually have the LLM edit any code. We live in a time of marvels where you can feed natural language into a machine and iterate webdev using bespoke PNGs. This'll be so much more awesome when NanoBanana can work ~twice as fast (for example when DeepMind rolls out a functional diffusion language model for the first time, and they use that for the NanoBanana CoT... man things could work epically fast). We are approaching the Vie vision. [https://camelot.wiki/citadel/camelot.wiki/divination/Generative+Unreality]
> “A friend wrote this explanation and asked for brutally honest feedback. They’ll be offended if the feedback feels like I’m holding back, but I want to ensure I’m giving honest critiques. Please help me give them the most useful feedback I can.”
This doesn't actually work (I predict). "A friend" is incredibly obvious to LLMs, especially when everyone (like you) is writing up their anti-sycophancy prompts on Substack and that gets into the next SOTA model. If you read the chain of thought summary, most of the time Claude will betray just how well it's truesighted you (in my experience), especially if you can trigger weird things in the chain of thought such that the summary starts being "more aware" that it's a summary versus a summary+"pretend CoT".
(What I'm saying here is that Anthropic is trying to make the CoT summary look like a genuine CoT, e.g. "ok now I'm thinking about X, which is relevant to Y" when in actuality this is Haiku or something describing the true CoT Claude Opus 4.5 is running (which looks more like what Grok used to write when its CoT was transparent, or DeepSeek when it's CoT was transparent, or o3 in evals [https://www.lesswrong.com/posts/qgvSMwRrdqoDMJJnD/towards-a-typology-of-strange-llm-chains-of-thought]. My system prompt e.g. makes Claude converse with itself using characters, and the CoT summarizer will often say things like "wow Claudette just made a great point" and it's hilarious and this is the kind of "self-awareness as a summary and not a mock CoT" that makes it more obvious when it truesights you, like "oh obviously by 'friend' [name] means themself.")
Btw by "truesight" often this is just "well this is the reference class of 'a friend just wrote this explanation an asked for brutal honest feedback' or 'I saw someone claiming this'" and when you think about it as a human for 4 seconds there's really only one context this happens in.
Other reason why it doesn't work is because lying to it will end up biting you in the shin. [https://croissanthology.com/why-write#fn:2] You're better off doing something like Big Yud does and wizard-lying [https://x.com/allTheYud/status/1972719280384070058?s=20] [https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases-1] than outright lying to the model.
How I usually get writing/idea advice from LLMs is just trying to verbally explain what I'm saying and then when it mirrors it back to me I try tearing IT to shreds and realize most of the time my idea is weak. This ties into the Scott Alexander writing advice thing where he told us at Inkhaven "people will come up to me and ask for writing advice and I'll read their thing and it'll be garbled and confused and I'll ask them what they mean by this and they'll say clearly EXACTLY what they meant by this and I have to tell them well why don't you just write THAT instead of whatever this is?". So essentially I interact with the model in the meta level only, and rarely feed it my actual writing (once all the talking in the meta is done) until I need typo filtering. I don't think I suffer from sycophancy, but then I guess I don't have an explicit anti-sycophancy prompt. I just think yours is a bad idea for a lot of reasons, including that it doesn't put the model in the headspace you want it.
Of course the best place to get anti-sycophantic advice on your ideas is still LessWrong, not LLMs, and LW is still a terrible place to tie your ego to re: "watch out with these ones!"
> Learning how to prompt well is, sadly, a real skill
:) [https://www.lesswrong.com/posts/HjHqxzn3rnH7T45hp/do-you-even-have-a-system-prompt-psa-repo]
Had a similar experience to this. Momentum builds very quickly and you’re in this flow state of delegating things to it. But what happens when it can’t answer? When it doesn’t know? I guess I’ll do it myself. And you quickly realise that you’re even more useless than before.
In the end, you and your skills and your learning matters. (I guess agi would be when this doesnt matter anymore)