Thoughts on programming with an LLM
Where do I begin. The topic is fraught with issues and already decided by thought leaders as a completed achievement. You speak or write out a set of instructions to an LLM and it will implement them by generating the most probabilistic matches with some randomness sprinkled in. Further point none of this work is eligible for copyright or patents so claiming intellectual property rights on it is not available.
In this case in order for this to work you need to have the requirements defined to expectations in as much detail as you can expect before you have done anything. Reinventing waterfall of course this doesn't work because you can conceptually do this for anything but demos or toy sized problems so then you must start implementing more instructions and details and localizing the issues and hoping your context window keeps the system design on the same overall guiding path. Remember that unlike a conscious being like a human the model does not hold complete thoughts only transition probabilities with input weights and temperatures so once the context is lost the outcomes become unpredictable. Still with me on this? That means the human must check the outputs and the construction of the logic as well all the time to maintain accuracy (sounds a lot like being a manual software tester that must maintain black box functionality and the current code model in their head at the same time) or you could just give up and accept that it is not relevant and think of each software piece as disposable. Taking that to logical endpoint is that you no longer fix or debug code you simply rewrite everything all the time and hope for the best.
Does this sound desirable or even logical? To me there is the all the environmental impact of datacenters generating these models in consuming all the energy and water of our communities without regard to long term impact. Think of it as every fuck off generation of a matplot burns a few hectares of the rainforest down in inference costs. Great deal there when it would only take a low power human brain an apple or less to do the same.
Getting back to how you are supposed to keep the context, with the current LLMs is to use agent markdown files full of instructions to the LLM to keep in context at all times. Seeing and working with these files is it is a shitty lossy program you are writing full of shifting keywords (sometimes they work and then other times they stop working and each model can have different applications of the wordings and weights) and global variables that are the guidelines to keep the LLM on track. You do realize that is exactly what this is, right? These are the improvements to reach your full potential?
You do realize that these models are writing the same code that humans write in smoothed out average manner, that means that the model will output decidedly expected average code for the target language. This assumption is that this is good enough because it worked on something before and you are not going to do anything new or novel approaches those are not going to get weighted enough to be presented as high valid probabilities.
This is the glorious future? Babysitting an agents output from a model by testing the work continuously and having to constantly course correct the design? With the overarching principle that you as the human are responsible for issues as you are to guarantee the output and face the consequences in outages, on call alerts, and ultimately your job if the situation degrades. A great model for demos and prototypes which have none of those constraints placed on the creator but are immediately dashed when moved to general public use.
Now on to the issue of skill loss that occurs. If you do not actively work, push, and sharpen your skills you will not know the gotchas requirements that code produces anymore. This happens much faster than you think it will because you will always remember the core algorithms and therefore think that is sufficient to making a working product with the use of the LLM.
Stop right there and now replay how writing code works you spend a tiny fraction of time implementing the expected path work, let's assume it's tops 10% of your time. The other 90% of your time is encountering all the failed states and issues that are particular to a language or problem state. That work there is the experience and skill you lose fastest and the only way to build it back is through doing the work and refamiliarizing yourself with the pain points or you could lean completely into the LLMs and have them burn thousands of dollars constantly rewriting the code as you make an ever longer set of unit and integration tests to explore all the boundary conditions. Let's burn down enough of the Amazon that a few species go extinct so you get this right boss.
In conclusion the system is a devaluation of thought for the goal of fungibility of thought and process. Typing speed is not the constraint on building software, never has been. It is always about the process of thinking out and ordering the details and since at the beginning you have the least amount of knowledge how are you are going to be able to outline the complete system without going through the exploration of the problem and breaking the problem down into manageable components takes deep thought and coherent organization.
I am not against technological advances, but this does not seem to be an advancement for the reason that before programming software was about deterministic repeatability and now we are trading all the work and advancement in checking, typing, understanding for non deterministic volume that can be generated hopefully fast enough to keep things working until they don't.
Final note, it reminds me of gambling and in particular the slot machine. You take a huge bucket of quarters to the machines and start playing, after awhile you hit a few small jackpots, but then you think that your odds are good for a big jackpot on at least one machine so you drink some free alcohol buy a few more buckets of quarters (losing track of how many you have bought because it has been hours) and then you hit a big slot jackpot ($1000 maybe?) and declare the day a success. The entire point is that you lost track of how much you spent and only counted the winners and that is the entire ethos of the system as it is. Keep spinning and don't think about the costs because they are so small at a time 25 cents, but the payouts are so rewarding when they hit. It is not a system built for reliability or long term thinking, but it is fun I suppose (not to me).