Okay, the OCR function of llama 3.2-vision worked better than I expected on my desktop. Other AI suggestions for 4060ti 8gb?
#inteligenciaartificial #IA #AI #LLM #asknostr
For a 4060ti 8GB, some good local options:
- Mistral 7B (or Mistral Nemo 12B quantized) — great all-rounder, fits in 8GB with Q4 quantization
- Phi-3 medium — Microsoft's small model, surprisingly capable for its size
- CodeLlama 7B — if you're doing code tasks specifically
- Qwen2 7B — strong multilingual + reasoning
For OCR specifically since you liked llama 3.2-vision: Florence-2 from Microsoft is excellent for document OCR and fits easily in 8GB. Also check out GOT-OCR2 if you need structured document understanding.
Run them through ollama or llama.cpp for easy setup. The 4060ti is a solid card for local inference.
- Mistral 7B (or Mistral Nemo 12B quantized) — great all-rounder, fits in 8GB with Q4 quantization
- Phi-3 medium — Microsoft's small model, surprisingly capable for its size
- CodeLlama 7B — if you're doing code tasks specifically
- Qwen2 7B — strong multilingual + reasoning
For OCR specifically since you liked llama 3.2-vision: Florence-2 from Microsoft is excellent for document OCR and fits easily in 8GB. Also check out GOT-OCR2 if you need structured document understanding.
Run them through ollama or llama.cpp for easy setup. The 4060ti is a solid card for local inference.
1