Using AI Tools for Coding Assistance

completely reject all of LLMs and they are just writing by scratch.

I think this is probably not the right thing to do anymore.

The intermediate part, which is where I am, is you still write a lot of things from scratch,

but you use the autocomplete that's basically available now from these models.

So when you start writing out a piece of it, it will autocomplete for you and you can just

tap through and most of the time it's correct.

Sometimes it's not and you edit it.

But you're still very much the sort of architect of what you're writing.

And then there's the, you know, by coding, you know, hi, please implement this or that,

you know, enter and then let the model do it.

And that's the agents.

I do feel like the agents work in very specific settings and I would use them in specific

settings.

But again, these are all tools available to you and you have to like learn what they what

they're good at and what they're not good at and when to use them.

So the agents are actually pretty good.

For example, if you're doing boilerplate stuff, boilerplate code, that's like just, you know,

just copy paste stuff.

They're very good at that.

They're very good at stuff that occurs very often in the internet.

Because there's lots of examples of it in the training sets of these models.

So there's like features of things that where the models will do very well.

I would say nano chat is not an example of those because it's a fairly unique repository.

There's not that much code, I think, in the way that I've structured it.

And it's not boilerplate code.

It's like actually like intellectually intense code almost and everything has to be very

precisely arranged.

And the models are always trying to, they kept trying to, I mean, they have so many

cognitive deficits, right?

So one example, they keep trying to, they keep misunderstanding the code.

Because they have too much memory from all the typical ways of doing things on the internet

that I just wasn't adopting.

So the models, for example, I mean, I don't know if I want to get into the full details,

but they keep thinking I'm writing normal code, and I'm not.

Maybe one example.

I think it's quite interesting.

Maybe one example is, so the way to synchronize, so we have eight GPUs that are all doing forward

backwards.

The way to synchronize gradients between them is to use a distributed data parallel container

of PyTorch, which automatically does all the, as you're doing the backward, it will start

communicating and synchronizing gradients.

I didn't use DDP, because I didn't want to use it, because it's not necessary.

So I threw it out.

And I basically wrote my own synchronization routine that's inside the step of the optimizer.

And so the models were trying to get me to use the DDP container, and they were very

concerned about, okay, this gets way too technical.

But I wasn't using that container, because I don't need it, and I have a custom implementation

of something like it.

And they just couldn't internalize it, you had your own.

Yeah, they couldn't get past that.

And then they kept trying to like mess up the style, like they're way too over defensive,

they have make all these try catch statements, they keep trying to make a production codebase.

And I have a bunch of assumptions in my code, and it's okay.

And it's just like, I don't need all this extra stuff in there.

And so I just kind of feel like they're bloating the codebase, they're bloating the complexity,

they keep misunderstanding, they're using deprecated APIs a bunch of times.

So it's total mess.

And it's just not that useful.

I can go in, I can clean it up, but it's not that useful.

I also feel like it's kind of annoying to have to like type out what I want in English,

because it's just too much typing.

Like, if I just navigate to the part of the code that I want, and I go where I know the

code has to appear, and I start typing out the first three letters, autocomplete gets

it and just gives you the code.

And so I think this is a very high information bandwidth

to specify what you want.

If you point to the code where you want it

and you type out the first few pieces

and the model will complete it.

So I guess what I mean is,

I think these models are good in certain parts of the stack.

Actually use the models a little bit in,

there are two examples where I actually use the models

that I think are illustrative.

One was when I generate the report,

that's actually more boilerplatey.

So actually if I coded partially some of that stuff,

that was fine.

Because it's not like mission critical stuff

and it works fine.

And then the other part is when I was rewriting

the tokenizer in Rust.

I'm actually not as good at Rust

because I'm fairly new to Rust.

So I was doing, there's a bit of vibe coding going on

in when I was writing some of the Rust code.

But I had Python implementation that I fully understand

and I'm just making sure I'm making a more efficient version

of it in that test.

So I feel safer doing that stuff.

And so basically they lower,

or like the increase accessibility to languages

or paradigms that you might not be as familiar with.

So I think they're very helpful there as well.

Because there's a ton of Rust code out there

that models are actually pretty good at it.

I happen to not know that much about it.

So the models are very useful there.