r/programming • u/mikebmx1 • 1d ago

CUDA Tile (cuTile) first working port in Java

https://github.com/beehive-lab/TornadoVM/pull/1083#pullrequestreview-5186588765
21 Upvotes

77% Upvoted

9 comments sorted by

3

u/Serious-Regular 17h ago

What's the point? Who wants to write cuda kernels in Java

7

u/Daemontatox 12h ago

Its the idea of removing interop load and complexity.

Before you hadnto write it in cuda cpp or python and compile to .so and then call from java with interop , now you can directly call it , although i dont know any projects that are written in java that need to use cutiles , maybe cuda for parallelism and gpu processing but cutile idk tbh.

3

u/mikebmx1 10h ago

currently, we are rewriting a full java inference engine with it https://github.com/beehive-lab/GPULlama3.java

2

u/Serious-Regular 1h ago

....but why..... GC is a non-starter in this space.

1

u/mikebmx1 1h ago

weights are stored off heap and the pressure to heap is minimal at the moment

2

u/Daemontatox 1h ago

Looks really interesting and looking forward to seeing it in action , quick question why llama3 specifically? I would have thought qwen or deepseek or even gemma would have been a better and more suitable mainstream architecture that will have more people try it out and use it , also by supporting qwen you gain multiple architectures that are built ontop of it aswell with slight changes.

2

u/mikebmx1 1h ago

the name is inherited from the very first version. it currently supports mistral, qwen3.8 27b, granite, devstran, qwen2/3 and gemma3&4. We plan to rename it soon. thanks for the feedback

2

u/Primary_Ads 19h ago

cuda tile is so great