Are there any LLMs that can run with decent performance on hardware comparable to Jetson NX?

Models
3

Hello, I think we can help you about the memory bottleneck.

Please check the post I uploaded on the forum last week. If you are interested, please give us a message via our homepage. We would like to discuss details!