1. Forum
  2. Usenet
  3. COMP.LANG.MODULA2

  • New Modula-2 M-Code Interpreter

    From Guido@21:1/5 to All on Fri Apr 29 13:50:14 2022
    Inspired by Jos Dreesen's Lilith emulator and following his recent discovery of the ETHZ M2 single pass compiler sources, I decided to build my own take of a runtime environment for the compiler and the M-code object files it generates.

    Instead of emulating the Lilith machine on a microcode level, my design interprets the M-code opcodes directly. Also, several core functions of Lilith's original Medos-2 operating system are carried out by the interpreter itself, which allows the design
    to become less dependent on the original Lilith machine architecture and also improves performance. The main areas concerned are object file loading and staging, memory management and OS-level file and device management.

    The bottomline of this approach (and the most pronounced difference to Jos Dreesen's emulator) is that It no longer faithfully simulates the historic Lilith machine. In particular, the interpreter (currently) lacks the bit-mapped graphic capabilities of
    the original machine. On the plus side however, it is now possible to run (character-based) Modula-2 programs in a standard terminal environment. Also, we can read/write files in the UNIX file system directly. And finally, code and data no longer share
    the same limited address space as in the Lilith, although this advantage might be considered theoretical at best in 2022 :)

    The interpreter is "Wirth Level 1 Compliant", which I define as the ability to execute N. Wirth's single pass compiler to compile its own sources :) The bootstrapping process to get there relied heavily on Jos' emulator, which I used to build the single
    pass compiler with the earlier multipass compiler (not so easy, by the way!). Also, I needed to adapt the initial set of M2 libraries required by the compiler to the specific environment of my interpreter before reaching the pivotal point where
    everything could compile itself under my interpreter.

    (In the process, I could not resist to tweak Wirth's compiler to read files with standard UNIX LF characters, something which might have been considered a heresy back in the day :) )

    Everything is still work in progress and there are plenty of rough edges, but you can check out a beta version at

    https://github.com/ghoss/m2emul

    The setup includes the interpreter, a minimal runtime environment with M2 libraries and the original single pass compiler, as well as my own very primitive command shell inspired by the "SEK.Comint" command line interpreter of Lilith's original Medos-2
    OS.

    Cheers,
    Guido

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jos Dreesen@21:1/5 to Guido on Sat Apr 30 10:32:42 2022
    The interpreter is "Wirth Level 1 Compliant", which I define as the ability to execute N. Wirth's single pass compiler to compile its own sources :) The bootstrapping process to get there relied heavily on Jos' emulator, which I used to build the
    single pass compiler with the earlier multipass compiler (not so easy, by the way!).


    There has been available, from Emulith's beginning, a binary of the single-pass compiler . Would it not have been simpler to use that ?

    Otherwise, nice development !

    Jos

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From nemo@21:1/5 to Guido on Sat Apr 30 14:48:03 2022
    On 2022-04-29 16:50, Guido wrote:
    Inspired by Jos Dreesen's Lilith emulator and following his
    recent discovery of the ETHZ M2 single pass compiler sources, I
    decided to build my own take of a runtime environment for the
    compiler and the M-code object files it generates.

    Instead of emulating the Lilith machine on a microcode level, my
    design interprets the M-code opcodes directly. Also, several core
    functions of Lilith's original Medos-2 operating system are
    carried out by the interpreter itself, which allows the design to
    become less dependent on the original Lilith machine architecture
    and also improves performance. The main areas concerned are
    object file loading and staging, memory management and OS-level
    file and device management.

    The bottomline of this approach (and the most pronounced
    difference to Jos Dreesen's emulator) is that It no longer
    faithfully simulates the historic Lilith machine. In particular,
    the interpreter (currently) lacks the bit-mapped graphic
    capabilities of the original machine. On the plus side however,
    it is now possible to run (character-based) Modula-2 programs in
    a standard terminal environment. Also, we can read/write files in
    the UNIX file system directly. And finally, code and data no
    longer share the same limited address space as in the Lilith,
    although this advantage might be considered theoretical at best
    in 2022 :)
    [...]

    Everything is still work in progress and there are plenty of rough edges, but you can check out a beta version at

    https://github.com/ghoss/m2emul
    [...]
    Cheers,
    Guido


    I look forward to seeing this good work completed.

    As a side issue, Andreas Borchert did a fair amount of work that
    included a Lilith emulator, linker, and others tools. His work is here: https://github.com/afborchert/lilith (Caveat is that his C source is
    K&R1 but there are tools to correct that.)

    N

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Guido@21:1/5 to Jos Dreesen on Sun May 1 03:42:43 2022
    On Saturday, 30 April 2022 at 10:32:43 UTC+2, Jos Dreesen wrote:
    The interpreter is "Wirth Level 1 Compliant", which I define as the ability to execute N. Wirth's single pass compiler to compile its own sources :) The bootstrapping process to get there relied heavily on Jos' emulator, which I used to build the
    single pass compiler with the earlier multipass compiler (not so easy, by the way!).
    There has been available, from Emulith's beginning, a binary of the single-pass compiler . Would it not have been simpler to use that ?

    Otherwise, nice development !

    Jos

    I do realize that, but keep in mind that this is a linked binary which includes baked-in versions of low-level libraries like "FileSystem", "Storage", "Monitor" etc. – These implementations refer to the original Lilith machine (and disk) architecture
    and thus were not guaranteed to run 1:1 in the environment simulated by my interpreter. Besides, my interpreter doesn't use linked binaries at all; if a referenced module has been loaded by the system earlier, that instance is used.

    Also, slightly different versions of these low-level libraries exist on several Lilith disk images, so I wasn't really sure what Wirth actually used to develop his compiler. Since the discovered source code doesn't compile straightforward with the
    earlier multipass compiler, there must have been unknown precursors and earlier stages of the source. As a result, I felt it would be safer to start from the source code instead of the binary.

    However, now that I know about the dependencies and workings of the single-pass compiler, it should be feasible to write a simple "unlinker" (possibly true to form in Modula-2) which decomposes linked binaries back into their original modules. Then, any
    incompatible original modules (such as Storage etc.) could be swapped out with the versions I rewrote. This patched compiler binary (and possibly other interesting binaries with missing source code) should then work with my interpreter. In fact, I might
    actually do just that as a fun exercise, since the original binary is dated 1987 and thus more recent, while the discovered sources seem to be from 1986.

    Guido

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Guido@21:1/5 to nemo on Sun May 1 03:43:36 2022
    On Saturday, 30 April 2022 at 20:48:05 UTC+2, nemo wrote:
    I look forward to seeing this good work completed.

    As a side issue, Andreas Borchert did a fair amount of work that
    included a Lilith emulator, linker, and others tools. His work is here: https://github.com/afborchert/lilith (Caveat is that his C source is
    K&R1 but there are tools to correct that.)

    N

    Yes, I did mention Borchert in the credits ;) While both of us obviously started from the same reference material (the more-or-less complete M-code definition by ETHZ in the form of an "interpreter" written in Modula-2), Borchert's interpreter clearly is
    more faithful to the source, while my version takes some "artistic license" and departs from the Lilith machine architecture in some aspects. For example, module tables and code frames are managed and stored in separate structures by the interpreter and
    do not form part of the main memory space, thus allowing for greater control over what's going on during execution. Also, Borchert did not need to implement all M-codes (such as the floating point functions of opcode FFCT) for his specific purpose at the
    time. These are not even defined in the ETHZ papers in a useful manner and took me some time to reverse-engineer, especially since they only ever seemed to be used in one place (module "MathLib0").

    In many places where the original M-code definition was ambiguous or incomplete, Borchert's code (along with the Soviet Kronos paper) proved invaluable for making sure that I actually was on the same page. In the end however, I felt that a rewrite from
    scratch would be safer than an attempt at porting and converting his code given the ancient K&R compiler and execution-order issues which he mentioned himself… but most importantly, I wanted to build my own Lego brick house :)

    Guido

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)