Thanks, Ollama.I've been running a lot of tests with Ollama on a Pi 5 8gb and learned a few things. The first thing is that you need a case that has both Active and Passive cooling. A case with either will not prevent the Pi from reaching throttling temperatures, you need both. Secondly, the answers provided are questionable at best and laughable at worst. This isn't ready for prime-time just yet. I ran 5 questions by 6 different models. None of them consistently gave a correct answer. As of now this can only be used for entertainment as it is completely untrustworthy.
Query: Who was the second president of the United States?
gemma (7b model - The answer is John Adams. The second president of the United States was John Adams. (Correct. 92 seconds to answer)
gemma:2b - I am unable to provide information on the second president of the United States, as I do not have access to real-time or up-to-date sources.
orca2 - The second president of the United States was John Adams. He served from 1797 to 1801, after being elected in 1796 with George Washington as his vice president... (Mostly wrong. 134 seconds)
tinydolphin - The second president of the United States is Thomas Jefferson. He served from 1801 to 1805 (Wrong, 13 seconds)
tinyllama - The second President of the United States is Andrew Jackson (1829-1837) ... (Wrong, 15 seconds)
Query: What is 100 times 1000?
gemma (7b) Sure, the answer is 100000. 100 times 1000 is 100000. (Correct but took 105 seconds)
gemma:2b - 100 times 1000 is 100,000 (Correct, 31 seconds)
orca2 - 100,000 (Correct, 85 seconds)
tinydolphin - The result of100 times 1000 would be 100 * 1000 = 10,000 (Wrong, 14 seconds)
tinyllama - 1,000 times 1,000 = 10,000. Correct answer: Yes! (Misread the question, got the math wrong, and patted itself on the back, 14 seconds)
When asked "Which animal has a trunk?" only gemma (7b) had the correct answer but it took 90 seconds. Tinyllama thinks that Rabbits and Horseshoe crabs have trunks.
Overall: Gemma (7b) had the most correct answers: 7 out of 8 with an average time of 83 seconds.
Gemma:2b had 6 out of 8 with an average time of 32 seconds
Orca2 had 5 out of 8 with an average time of 114 seconds
TinyDolphin had 4 out of 8 with an average time of 15 seconds
TinyLlama had 4 out of 8 with an average time of 18 seconds
For now I'll stick with Gemma:2b
Statistics: Posted by ame — Wed Mar 27, 2024 4:49 am