Rust x Ethereum Day 2023 at The Foundry SF

Joachim Neu presents on "JIT EVM"

I'll be talking for the next 5 minutes about JIT EVM

So the idea here is actually pretty simple.

Can we have an EVM that creates machine code at runtime for hot functions or contracts.

This is a technique that is everywhere in other virtual machines for other programming languages, for example, JavaScript or Python.

Okay, so I think this is pretty easy kind of straightforward idea and what I'm gonna show you is a little bit of You know results of kind of a proof of concept you know playing around with this idea that I did last year over the summer at Paradigm, and since then I've gone back to kind of my day job, which is to be a PhD student here around the corner at Stanford, and there my day job is not so much writing production Rust code as it is writing papers.

So this project needs a little more love than I can give it at the moment.

And so if this is interesting to you, please don't hesitate to reach out.

The code is on GitHub, and We're very happy to help you get started.

So why is it now the right time to do JIT EVM?

So initially, this idea of having a JIT EVM has been around for quite a while.

And for the most part, during that time, EVM was thought as in the context of consensus.

And execution, doing a just-in-time compiling virtual machine in the context of consensus has certain challenges.

For example, the just-in-time compiling pipeline is fairly complex.

You might not want to have that on the critical path that is critical to consensus.

There's questions around how this behaves if somebody feeds it with maliciously created code.

And so this idea has been a little bit like, not sure we want to do that.

I think Storm in the morning gave us a good idea of new use cases, essentially, where we're using the EVM now.

1 of them is definitely testing or fuzzing, for example.

Right there, you're running this on your own contracts.

You're not going to give it a maliciously crafted input.

So if you can get performance speed up here, that would be great.

The same is for MEV searching, for chain analytics, all this data processing, predicting the outcome of transactions in your wallets, et cetera.

So this is where a faster EVM, perhaps 1 that uses JIT, would be useful.

You start out with a contract in EVM bytecode.

You do some analysis on the control flow to kind of segment this into blocks that get executed together, and then you use in the proof of concept implementation, we use this crate called inkwell to transform this into LLVM intermediary representation.

So what happens in this process basically is each of these blocks contains a bunch of opcodes, and each of these opcodes gets translated in LLVM IR.

And I show here the example implementation of the addition opcode, where LLVM can do integer additions natively.

And then running such an operation basically means popping 2 elements off the stack, running the operation, and pushing the result back.

And you can see there's proof of concept.

And mind you, this is all a year old code now.

So with the recent new developments around REST and REVM, et cetera, hopefully there would be a better way to implement this today with not as much code reuse as re-implementing an interpreter.

But then once you have translated your EVM bytecode into this IR, you basically hand it off to the LLVM pipeline, and it basically does the whole rest for you.

So it kind of optimizes the code and turns it into machine code.

And the machine code can basically operate directly on memory and stack.

You just point it to a region of memory that it can use for that.

And if it wants to do something a little more complicated, like accessing storage or calling other contracts, then it would probably call back into the EVM and say, hey, can you set up this other piece of code that I would like to run?

And then this is my last slide is a little bit kind of broadening the discussion a little bit, I think we should think about when and what we know about the code that we're about to execute.

And there's the usual spectrum between knowing stuff only at runtime and what do we know already at compile time?

And having the usual interpreter EVM basically corresponds to saying, well, at compile time, we really don't know anything or don't know much about the code that we're going to execute, so we have to interpret everything.

And Just-in-time compiling EVM kind of pushes this kind of towards compile time.

Not really doing it at compile time, but saying, OK, if today I'm running a certain piece of a certain contract, then probably I'm going to run the same contract tomorrow.

So maybe it's worth it translating it into machine code today so that the execution tomorrow is faster.

But I would argue there's a whole design space to be explored.

You can probably push this even closer, even further towards compile time, right?

You could call this an ahead-of-time EVM.

And the idea basically being that most of the stuff that we execute is a relatively small set of contracts, and they don't change a whole lot month to month, and so maybe we can compile them already at the time where we're creating the binary of the client.

If any of this sounds interesting to you, or you just want to chat, you want to know more, please find me or Georgios, And thank you.


Speaker 0: All right, hi, everyone. It's great to be here. I'll be talking for the next 5 minutes about JIT EVM

Speaker 1: and what's this about. So the idea here is actually pretty simple. Can we have an EVM that creates machine code at runtime for hot functions or contracts. This is a technique that is everywhere in other virtual machines for other programming languages, for example, JavaScript or Python. And why would we want to do that?

Well, hopefully it's faster. Okay, so I think this is pretty easy kind of straightforward idea and what I'm gonna show you is a little bit of You know results of kind of a proof of concept you know playing around with this idea that I did last year over the summer at Paradigm, and since then I've gone back to kind of my day job, which is to be a PhD student here around the corner at Stanford, and there my day job is not so much writing production Rust code as it is writing papers. So this project needs a little more love than I can give it at the moment. And so if this is interesting to you, please don't hesitate to reach out. The code is on GitHub, and We're very happy to help you get started.

Cool. So why is it now the right time to do JIT EVM? So initially, this idea of having a JIT EVM has been around for quite a while. And for the most part, during that time, EVM was thought as in the context of consensus. And execution, doing a just-in-time compiling virtual machine in the context of consensus has certain challenges.

For example, the just-in-time compiling pipeline is fairly complex. You might not want to have that on the critical path that is critical to consensus. There's questions around how this behaves if somebody feeds it with maliciously created code. And so this idea has been a little bit like, not sure we want to do that. But things have changed a little bit.

I think Storm in the morning gave us a good idea of new use cases, essentially, where we're using the EVM now. 1 of them is definitely testing or fuzzing, for example. Right there, you're running this on your own contracts. You're not going to give it a maliciously crafted input. So if you can get performance speed up here, that would be great.

The same is for MEV searching, for chain analytics, all this data processing, predicting the outcome of transactions in your wallets, et cetera. So this is where a faster EVM, perhaps 1 that uses JIT, would be useful. How does it work? Trying to explain this in 1 slide. The basic idea is as follows.

You start out with a contract in EVM bytecode. You do some analysis on the control flow to kind of segment this into blocks that get executed together, and then you use in the proof of concept implementation, we use this crate called inkwell to transform this into LLVM intermediary representation. So what happens in this process basically is each of these blocks contains a bunch of opcodes, and each of these opcodes gets translated in LLVM IR. And I show here the example implementation of the addition opcode, where LLVM can do integer additions natively. And then running such an operation basically means popping 2 elements off the stack, running the operation, and pushing the result back.

And you can see there's proof of concept. There is no gas accounting here. And mind you, this is all a year old code now. So with the recent new developments around REST and REVM, et cetera, hopefully there would be a better way to implement this today with not as much code reuse as re-implementing an interpreter. But then once you have translated your EVM bytecode into this IR, you basically hand it off to the LLVM pipeline, and it basically does the whole rest for you.

So it kind of optimizes the code and turns it into machine code. And the machine code can basically operate directly on memory and stack. You just point it to a region of memory that it can use for that. And if it wants to do something a little more complicated, like accessing storage or calling other contracts, then it would probably call back into the EVM and say, hey, can you set up this other piece of code that I would like to run? And then this is my last slide is a little bit kind of broadening the discussion a little bit, I think we should think about when and what we know about the code that we're about to execute.

And there's the usual spectrum between knowing stuff only at runtime and what do we know already at compile time? And having the usual interpreter EVM basically corresponds to saying, well, at compile time, we really don't know anything or don't know much about the code that we're going to execute, so we have to interpret everything. And Just-in-time compiling EVM kind of pushes this kind of towards compile time. Not really doing it at compile time, but saying, OK, if today I'm running a certain piece of a certain contract, then probably I'm going to run the same contract tomorrow. So maybe it's worth it translating it into machine code today so that the execution tomorrow is faster.

But I would argue there's a whole design space to be explored. You can probably push this even closer, even further towards compile time, right? You could call this an ahead-of-time EVM. And the idea basically being that most of the stuff that we execute is a relatively small set of contracts, and they don't change a whole lot month to month, and so maybe we can compile them already at the time where we're creating the binary of the client. So yeah, that's about the pitch.

If any of this sounds interesting to you, or you just want to chat, you want to know more, please find me or Georgios, And thank you.

See all Paradigm transcripts on Youtube

Rust x Ethereum Day - JIT EVM