230B parameters is “reasonably small” now?!

When you use the KDE Desktop Environment it will also show notifications from your phone and sync media controls (e.g. stop playback when you get a phone call). It’s great!

You can still leverage knowledge from a foundation model in a smaller fine-tuned one.

So the model might have learned general OOP principles from Java but it then drops redundant parameters about specific conventions like AbstractFactoryBuilders when it specialises on a language like Python which has no notion of Interfaces.

Likewise real world knowledge might help distinguish between accounting and database transactions when writing a banking application but you don’t necessarily need your coding assistant to have memorised all the world cup winners since 1966.

These models are unwieldy so I think it makes a lot of sense to try and find ones that are tuned efficiently.