Inline Assembly.

[lachlansneff]

Since cranelift is soon to be a backend for rust, it will need to support inline assembly. There is no good way to solve this right now, since rust uses the llvm inline asm syntax right now. I'm making this issue so we can think about this in the long-term.

[71]

I just had an idea that's very simple, but also very "raw," but I suppose it would be pretty good for the time being.
What if we simply allowed inline bytes, with the added ability to interpolate Cranelift values?

For example:

01 C0      ; add eax, eax
0F AF $v1  ; imul eax, v1
C3         ; ret

Edit: Obviously there is the problem of Cranelift values being encoded either as registers or stack variables for instance, but maybe some annotations could be added for a value to be in a register, like $(v1:reg) or $(ebb2:rel8).

[lachlansneff]

I'd be all for this for the time being actually, with a small modification.

Treat a block of code as an ebb essentially. Say what inputs it gets and what they should be stored into (registers, stack offsets, etc). The actual code block would be completely opaque. This would prevent cranelift from needing a disassembler.

For example:

asm(eax: v1, ebx: v2) -> eax:
  01 c0          ; add eax,eax
  0f af c3       ; imul eax,ebx

or

asm(v1 -> eax, v2 -> ebx) -> eax:
  01 c0          ; add eax,eax
  0f af c3       ; imul eax,ebx

[lachlansneff]

I'd like to see what @sunfishcode has to say on this.

[sunfishcode]

Interesting idea. My first question is, how do you envision using this?

Looking at the idea itself, a factor to take into consideration is that machine encodings are really complex. A register doesn't typically end up being encoded in a byte; it's usually smaller than a byte, its position in its containing bytes will depend on which operand of the instruction it's for, and sometimes it requires other changes elsewhere in the instruction. Some examples:

$ cat t.s
movq (%rcx), %rcx # The encoding of %rcx depends on where it appears in the instruction!
movq (%rsi), %rcx # These use different REX prefix bits than the instructions below!
movq (%r11), %rcx
movq (%r12), %rcx # This needs a extra byte!
movq (%r13), %rcx # This needs an extra byte in a different way!
movq (%r14), %rcx
$ cc -c t.s
$ objdump -d t.o
...

I think we could still make this system work, if we defined a sufficiently elaborate manifest that could describe all the edits that one would need to make once one knows what registers and stack slots everything will be in. That'd be fairly elaborate, but it might still be simpler than fully parsing instructions from text.

[lachlansneff]

I'd use it for inline stuff on Nebulet. Like talking to io ports, etc.

[sunfishcode]

A few more thoughts here:

I missed this above: in @lachlansneff's variant above, the IR specifies all the registers, so it wouldn't actually require Cranelift to fill in register values. It'd literally be a block of bytes that simply requires certain values in certain fixed registers at input and output. And maybe a list of clobbers. That seems like it wouldn't be too hard to implement. That said, it might actually be too simple, in the sense that it'd work for very simple things, but would be difficult to evolve into something that does more.

If you're willing to specify all the register inputs and outputs yourself, it wouldn't be that much different to just put the code you want in a .s file and then call it, with a specialized calling convention if you want. Support for custom calling conventions is something we do have other uses for, so we wouldn't mind improving that.

Or, for in/out or similar instructions on x86, would it be enough if we exposed whatever instructions you need as instructions in Cranelift, so that you could use them, similar to compiler intrinsics? This would be easy to implement, and much easier for anyone writing an OS to use.

[nbp]

For your information, there is a rustc issue to stabilize how to do inline assembly. In particular, I suggested on this other thread a similar idea as @6a and @sunfishcode suggested, i-e. to have a set of bytes with constraints.

This idea got some push back on the basis that some people want to do inline assembly with a register allocation handled by the compiler, but I honestly do not see how this can be done without standardizing some form of assembly code.

rust-lang/rust#29722

[sunfishcode]

In case anyone wants to see where the idea above might go, here's a slightly more complete sketch. Here's a hypothetical IR structure for this form of inline asm:

struct InlineAsm {
    /// the raw bytes to start with
    data: vec<u8>,
    /// descriptions of all explicit input and output values
    constraints: vec<Constraint>, 
    /// extra registers, "memory", or other machine state which is clobbered
    clobbers: vec<String>,
    /// patches which can change or add bytes
    patches: vec<Patch>,
}

struct Constraint {
    // TODO: in/out/inout, tied, earlyclobber, hints, alternatives, register/memory/immediate classes, etc.
}

struct Patch {
    offset: u64, // byte offset in `data` *before* any patches are applied
    contents: PatchDetails,
}

enum PatchDetails {
    X86RexPrefix {
        w: bool, // REX.w
        inputs: vec<u64>, // indices in `constraints` for register operands which determine if a REX prefix is needed and if so what bits it should have.
    }
    X86ModRMRegField {
        input: u64, // index in `constraints` for the register to encode
    }
    X86ModRMRMField {
        input: u64, // similar
    }
    P2Align {
        // Insert fill bytes to align the following code to a 1<<p2align boundary.
        p2align: u8,
        // The byte value to insert.
        fill: u8
    }
    // TODO: lots more stuff here
}

I think this would deliver most of what LLVM/GCC-style inline asm do, assuming we filled it out. And it would eliminate the need for the backend to parse assembly text. That said, it's still fairly complex. And of course, no existing assemblers are built to work this way, including LLVM's assembler, so it would be a bunch of work to implement even in LLVM.

[lygstate]

Any idea? Inline assembly are essential for constructing OS or bare-metal environment.

[sunfishcode]

I'm not aware of any easy answers. Inline asm is a huge, complex, and nebulous set of features.

From a broader perspective, there's a form of an XY problem. If you ask for "inline asm", it may be years before we can deliver it. We want to help, but you're asking for a lot, and we don't know of any way to do it faster while still achieving our other goals. If you instead were to ask for access to certain machine instructions, or control registers, or ways to guide the register allocator around key pieces of code, or other specific things, plausible solutions sometimes could take just a few days to implement. With no exaggeration, more specific features can often be a thousand times easier to implement, and typically end up more robust, with more well defined interfaces, and better-understood implementations.

We may still implement inline asm eventually. People interested in seeing it happen are encouraged to get involved and help out!

[skyne98]

Hey, @sunfishcode, as far as I understand we cannot just allow using common register names like eax, ebx and stuff, right? It will require compiling the code with some of the already existing assembly compilers.

[nbp]

@skyne98 , you can and you should because some instructions require special registers. However, many uses of inline assembly are meant to provide an assembly template with which a register allocator can work with. For example, making a SIMD library which is using inline assembly does not want to hard-code a specific register allocation, as multiple consecutive uses of this inline assembly would cause too much register congestion that would penalize the performance of SIMD.

[skyne98]

So, what I mean is that the template you provide does not map directly to some binary instruction, it depends on things -- therefore you cannot just parse them, convert to binary and put into the "executable", right?

Otherwise, what is the complexity @sunfishcode was talking about? SimpleJIT is already "jitting" out some binary code.

[sunfishcode]

That's right. In general, the template cannot be parsed until the substitutions are made. And it may contain more than just instructions; it may use arbitrary assembler directives like .align or .section
or other things which have effects other than just contributing bytes to add to the output. And they may make use of arbitrary symbolic expressions that need to be evaluated and/or patched in later. They may rely on numerous historical assembler parsing quirks. They may use complex and interdependent operand constraints, and they may require the compiler's register allocator to find optimal solutions to those constraints. They may have ambiguous constraints like "memory" which ostensibly clobbers all memory but in practice compilers treat as not literally all memory. And so on.

[skyne98]

Also, @sunfishcode, what do you think about this project. To me seems like it can be at least partially helpful, however I have to admit I haven't read the source yet.

Also, I don't think we need all the features you mentioned above. What I was trying to ask before is that if doing substitutions is possible and not too hard to do, then introducing only adding basic instructions like stack pushes, pops and 'mov'-s (just add 20 most used instructions in kernels as well as some important ones, we may even try to analyze code for that) will help A LOT and make writing bare metal code at least possible to do.

What do you think?