Vibecoding an inference chip

(ankitg.me)

58 points | by ankitvgupta 3 hours ago ago

19 comments

  • cpldcpu an hour ago

    I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

    It seems implementing transformers is the new "hello world" of AI assisted coding.

  • datadrivenangel 2 hours ago

    Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.

    • risyachka an hour ago

      Fpga is software-driven, model can verify its output easily.

      Zero surprise it did it

    • datadrivenangel 2 hours ago

      it only cost them $10,000 in Claude API usage. :)

      • hmate9 2 hours ago

        You get that much usage in a monthly $200 subscription.

        • dvh an hour ago

          So who's paying the cost?

          • stronglikedan 22 minutes ago

            Everyone else that doesn't use the entirety of the plan they pay for.

          • nyc_data_geek 17 minutes ago

            Retail and society as ANT shares get pulled into index and wealth funds

          • ralph84 29 minutes ago

            API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.

          • exe34 an hour ago

            Corporate customers

        • dofm an hour ago

          And a kitten.

  • codekansas an hour ago

    This is cool

    Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

  • chalkycrimp an hour ago

    Good chance I'm not the only one thinking this: share the code

  • lxe 8 minutes ago

    Repetitive "make it faster" prompting is essentially what all these "RSI" claims are

  • spwa4 22 minutes ago

    I must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

    (unfortunately for a decent model it will also be price competitive with a big nvidia card)

  • saejox an hour ago

    i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.

  • Archit3ch 29 minutes ago

    Now do silicon. ;)