Fast GPT-2 inference written in Fortran | Hacker News Reader

Fast GPT-2 inference written in Fortran

github.com

83 pointsLoic13 comments

"Ondřej [Certik] is the original author of SymPy, SymEngine and LFortran".

https://www.sympy.org/en/index.html

https://lfortran.org/ << "Modern interactive LLVM-based Fortran compiler"

Raise your hand if you first encountered Fortran in (eng) school and wrote your first Fortran programs on punch cards. (MTS for bonus).

My school still made me take it in 2011... for a Mech-E.

Yes, besides the punch cards in 2013, superseded by a course in python at around 2016.

I can raise only one half of a hand as I started without punch cards, but we had cards at home because my mother "recycled" the unused one she add at the office to take notes.

(warily raises hand stained with ink from the diagonal lines drawn across the edges to ease collation of dropped decks)

Wonderful, I think I remember that! A great little hack right there.

        PRINT 10
  10    FORMAT(14HYOU'RE WELCOME)
        END

No punch cards, but I did write the Windows version of ITSM (software accompanying the Brockwell and Davis Time Series book). The while thing was written in Fortran, the event loop and calls to the Windows API for UI and graphics. It's still the best damn visualization of a series + periodogram I've ever come across (if I do say so myself).

2 months ago https://news.ycombinator.com/item?id=35159961 FastGPT: Faster than PyTorch in 300 lines of Fortran - 12 comments

Real fortran programmers can write fortran code in any language.

The author here. If you have any questions, let me know.

If there is anybody here who wants to help parallelize this, let me know!

The benchmark says the fastest model takes 0.3s for 20 tokens. Does this mean it would take 30 seconds for 2000 tokens?

So does the model simply extract the most likely answer to the prompt based on model weights?

Reply on news.ycombinator.com