I run a game when players can post messages, it's a game where players can kill each other, and people often send threats along the lines of "I will kill you". Telling Gemma that it should classify a message as game related or a real life threat, and that it is for a message in a game where players can kill each other and threats are a part of the game, and that it should mark it as game related if it is unclear if the message is a game related threat or a real life threat does not work well. For other similar tasks it seems to follow instructions well, but for serious topics it seems to be very biased, and often err on the side of caution, despite being told not to. Sometimes it even spits out some help lines to contact.

I guess this is because it was trained to be safe, and that affects it's ability to follow instructions for this? Or am I completely off here?

I find performance in resource constrained environments interesting.

In particular trying to find decent code models (on device backup) but also tts applications and voice to text.

i love the 27b model, and i use it basically daily. however when i tried to finetune it for a task in a low resource language, unfortunately i did not succeed: lora just did not picked up the gist of the task, full finetune lead to catastrophic forgetting.

may i ask four your advice, or do you have any general tips how to do that properly?

thanks in advance for your help :)

Would it be okay for you to divulge some more training information here? With 170M embedding parameters, how do you ensure no embedding collapse and keeping the embedding matrix stable at training time?

(i know i am asking too much, but just curious). There is a clear trade off for you with vocab / transformer layers. How did you arrive at the split of 170m/100m. Does this contribute to model's performance on task specific fine tuning? Any internal experiments you could share? or public info you could point us to? Anything would be amazing.

PS: I am sorry if this is rude, but this has so many decisions i am curious about. Not intending to undermine anything, this is amazing work, and thank you for the whole Gemma series.

If Gemini is going to repeat something at least its that the team is great, and not a disgrace!

What use-cases are a good fit for finetuning this model? More specific instruction following, knowledge from proprietary data, response tone?

I use Gemma27b currently with a custom agent wrapper and its working pretty well.

Or am I so far behind that "fine tuning your own model" is something a 12 year old who is married to chatGPT does now?

I get the sense that AI is at the “hobby kit computing” stage where they used to dump all the components in a box and give you a schematic and a soldering iron and happily say “you make it work!”

And that worked in the early days of computing because there was a small number of people really motivated for the outcome.

But fully assembled and packaged and tested in a nice looking box is where the real demand turned out to be.

I’m looking forward to the day Google doesn’t just dump a model and say “you do the rest”.

I want to fire up Ubuntu on a Strix Halo and say apt install then load the browser interface. Or just download and run a Mac installer and have it just work.

Arcane complex multi step build install configure processes for AI need to end in favor of one click install. I’m not interested in the process of making it run.

input: ``` Customer Review says: ai bought your prod-duct and I wanna return becaus it no good.

Prompt: Create a JSON object that extracts information about this customer review based on the schema given. ```

output: ``` { "type": "review", "class": "complaint", "sentiment": -0.853, "request": "return" } ```

So essentially just "making sense of" natural language such that it can be used in programmatic context. (among other applications of course)

To get good results, you probably need to fine tune this model to expected data very aggressively.

The idea is, if a 270MB model can do with fine tuning, why ship a 32GB generalist model?

My suggestions here are to hook this model up to a RAG system, then you can rely on an external knowledge store. Or you can try finetuning this model with the facts that are important to you, if you do that it should pick up that new knowledge quite quickly.

as mentioned in the blog post: > "it can execute tasks like text classification and data extraction with remarkable accuracy, speed, and cost-effectiveness."

In this case, I changed the prompt to:

---

Tallest mountains (in order):

```

- Mount Everest

- Mount K2

- Mount Sahel

- Mount Fuji

- Mount McKinley

```

What is the second tallest mountain?

---

Suddenly, it got the answer right 95+% of the time

Nice to see that the model is so up-to-date wrt. naming mountains.

Are they?

> This SVG code provides a clear and visually appealing representation of a pelican riding a bicycle in a scenic landscape.

You'll get better results.

That still requires more than 170 floppy disks for installation.

My first try:

user: "When was Julius Caesar born"

response: "Julius Caesar was born in **Rome**"

Beautiful :D

(I do not mean to detract from it - but it's just beautiful. It will require more effort to tame it.)

For sure it won’t generate great svgs, but it might be a really good conversational model

Could it be tamed with good role-system prompt crafting? (Besides fine-tuning.)

Think of Apple however you want, but they rarely ship bad/half-baked products. They would rather not ship a product at all than ship something that's not polished.

If you have the latest betas installed you can call those APIs right now.

They also support fine-tuning on the model that ships with ~every device:

https://developer.apple.com/apple-intelligence/foundation-mo...

Then Siri just kinda languishes for forever, and LLM's pass the torch of "Cool Tech", so they try and "Reinvigurate" the team, but with no clear direction. Are they going to be a cloud provider? Are they going to contract out the training? Are they gunna spin up a compute facility even after neglecting to do so since 2012?

Apple needs to just stop trying shit, and just get that app store money. That's why jobs appointed cook. Jobs new cook was no innovator, but he could make apple a money printing machine. Thats what they should stick with.

So only for a while, but it still counts :)

To answer this more precisely its a matter of choosing different data and training regimes and checking performance with evals.

And to make this fully concrete you're welcome to give it a try! Train this model on a taskset of your choice and measure the performance tradeoffs. You'll get a good sense of how LLM capabilities shift

Not sure how much data is needed to realistically fine-tune something like this and get useful output.

https://ai.google.dev/gemma/docs/core/huggingface_text_full_...