version 1.6, 2002/08/07 22:19:44
|
version 1.11, 2002/08/14 09:00:22
|
Line 1
|
Line 1
|
|
\input texinfo @c -*-texinfo-*- |
|
@comment %**start of header |
|
@setfilename vmgen.info |
@include version.texi |
@include version.texi |
|
@settitle Vmgen (Gforth @value{VERSION}) |
|
@c @syncodeindex pg cp |
|
@comment %**end of header |
|
@copying |
|
This manual is for Vmgen |
|
(version @value{VERSION}, @value{UPDATED}), |
|
the virtual machine interpreter generator |
|
|
|
Copyright @copyright{} 2002 Free Software Foundation, Inc. |
|
|
|
@quotation |
|
Permission is granted to copy, distribute and/or modify this document |
|
under the terms of the GNU Free Documentation License, Version 1.1 or |
|
any later version published by the Free Software Foundation; with no |
|
Invariant Sections, with the Front-Cover texts being ``A GNU Manual,'' |
|
and with the Back-Cover Texts as in (a) below. A copy of the |
|
license is included in the section entitled ``GNU Free Documentation |
|
License.'' |
|
|
|
(a) The FSF's Back-Cover Text is: ``You have freedom to copy and modify |
|
this GNU Manual, like GNU software. Copies published by the Free |
|
Software Foundation raise funds for GNU development.'' |
|
@end quotation |
|
@end copying |
|
|
|
@dircategory GNU programming tools |
|
@direntry |
|
* Vmgen: (vmgen). Interpreter generator |
|
@end direntry |
|
|
|
@titlepage |
|
@title Vmgen |
|
@subtitle for Gforth version @value{VERSION}, @value{UPDATED} |
|
@author M. Anton Ertl (@email{anton@@mips.complang.tuwien.ac.at}) |
|
@page |
|
@vskip 0pt plus 1filll |
|
@insertcopying |
|
@end titlepage |
|
|
|
@contents |
|
|
|
@ifnottex |
|
@node Top, Introduction, (dir), (dir) |
|
@top Vmgen |
|
|
|
@insertcopying |
|
@end ifnottex |
|
|
|
@menu |
|
* Introduction:: What can Vmgen do for you? |
|
* Why interpreters?:: Advantages and disadvantages |
|
* Concepts:: VM interpreter background |
|
* Invoking Vmgen:: |
|
* Example:: |
|
* Input File Format:: |
|
* Using the generated code:: |
|
* Changes:: from earlier versions |
|
* Contact:: Bug reporting etc. |
|
* Copying This Manual:: Manual License |
|
* Index:: |
|
|
|
@detailmenu |
|
--- The Detailed Node Listing --- |
|
|
|
Concepts |
|
|
|
* Front end and VM interpreter:: Modularizing an interpretive system |
|
* Data handling:: Stacks, registers, immediate arguments |
|
* Dispatch:: From one VM instruction to the next |
|
|
|
Example |
|
|
|
* Example overview:: |
|
* Using profiling to create superinstructions:: |
|
|
|
Input File Format |
|
|
|
* Input File Grammar:: |
|
* Simple instructions:: |
|
* Superinstructions:: |
|
* Register Machines:: How to define register VM instructions |
|
|
|
Simple instructions |
|
|
|
* C Code Macros:: Macros recognized by Vmgen |
|
* C Code restrictions:: Vmgen makes assumptions about C code |
|
|
|
Using the generated code |
|
|
|
* VM engine:: Executing VM code |
|
* VM instruction table:: |
|
* VM code generation:: Creating VM code (in the front-end) |
|
* Peephole optimization:: Creating VM superinstructions |
|
* VM disassembler:: for debugging the front end |
|
* VM profiler:: for finding worthwhile superinstructions |
|
|
|
Copying This Manual |
|
|
|
* GNU Free Documentation License:: License for copying this manual. |
|
|
|
@end detailmenu |
|
@end menu |
|
|
@c @ifnottex |
@c @ifnottex |
This file documents vmgen (Gforth @value{VERSION}). |
@c This file documents Vmgen (Gforth @value{VERSION}). |
|
|
|
@c ************************************************************ |
|
@node Introduction, Why interpreters?, Top, Top |
@chapter Introduction |
@chapter Introduction |
|
|
Vmgen is a tool for writing efficient interpreters. It takes a simple |
Vmgen is a tool for writing efficient interpreters. It takes a simple |
Line 12 it). The run-time efficiency of the res
|
Line 119 it). The run-time efficiency of the res
|
within a factor of 10 of machine code produced by an optimizing |
within a factor of 10 of machine code produced by an optimizing |
compiler. |
compiler. |
|
|
The interpreter design strategy supported by vmgen is to divide the |
The interpreter design strategy supported by Vmgen is to divide the |
interpreter into two parts: |
interpreter into two parts: |
|
|
@itemize @bullet |
@itemize @bullet |
Line 39 A @emph{virtual machine} (VM) represents
|
Line 146 A @emph{virtual machine} (VM) represents
|
machine code. Control flow occurs through VM branch instructions, like |
machine code. Control flow occurs through VM branch instructions, like |
in a real machine. |
in a real machine. |
|
|
In this setup, vmgen can generate most of the code dealing with virtual |
In this setup, Vmgen can generate most of the code dealing with virtual |
machine instructions from a simple description of the virtual machine |
machine instructions from a simple description of the virtual machine |
instructions (@pxref...), in particular: |
instructions (@pxref{Input File Format}), in particular: |
|
|
@table @emph |
@table @asis |
|
|
@item VM instruction execution |
@item VM instruction execution |
|
|
Line 60 source level.
|
Line 167 source level.
|
|
|
@item VM code profiling |
@item VM code profiling |
Useful for optimizing the VM insterpreter with superinstructions |
Useful for optimizing the VM insterpreter with superinstructions |
(@pxref...). |
(@pxref{VM profiler}). |
|
|
@end table |
@end table |
|
|
VMgen supports efficient interpreters though various optimizations, in |
@noindent |
|
Vmgen supports efficient interpreters though various optimizations, in |
particular |
particular |
|
|
@itemize |
@itemize @bullet |
|
|
@item Threaded code |
@item Threaded code |
|
|
Line 81 Replicating VM (super)instructions for b
|
Line 189 Replicating VM (super)instructions for b
|
|
|
@end itemize |
@end itemize |
|
|
As a result, vmgen-based interpreters are only about an order of |
As a result, Vmgen-based interpreters are only about an order of |
magintude slower than native code from an optimizing C compiler on small |
magnitude slower than native code from an optimizing C compiler on small |
benchmarks; on large benchmarks, which spend more time in the run-time |
benchmarks; on large benchmarks, which spend more time in the run-time |
system, the slowdown is often less (e.g., the slowdown of a |
system, the slowdown is often less (e.g., the slowdown of a |
Vmgen-generated JVM interpreter over the best JVM JIT compiler we |
Vmgen-generated JVM interpreter over the best JVM JIT compiler we |
Line 91 and all other interpreters we looked at
|
Line 199 and all other interpreters we looked at
|
interpreter). |
interpreter). |
|
|
VMs are usually designed as stack machines (passing data between VM |
VMs are usually designed as stack machines (passing data between VM |
instructions on a stack), and vmgen supports such designs especially |
instructions on a stack), and Vmgen supports such designs especially |
well; however, you can also use vmgen for implementing a register VM and |
well; however, you can also use Vmgen for implementing a register VM and |
still benefit from most of the advantages offered by vmgen. |
still benefit from most of the advantages offered by Vmgen. |
|
|
There are many potential uses of the instruction descriptions that are |
There are many potential uses of the instruction descriptions that are |
not implemented at the moment, but we are open for feature requests, and |
not implemented at the moment, but we are open for feature requests, and |
Line 101 we will implement new features if someon
|
Line 209 we will implement new features if someon
|
list above is not exhaustive. |
list above is not exhaustive. |
|
|
@c ********************************************************************* |
@c ********************************************************************* |
|
@node Why interpreters?, Concepts, Introduction, Top |
@chapter Why interpreters? |
@chapter Why interpreters? |
|
|
Interpreters are a popular language implementation technique because |
Interpreters are a popular language implementation technique because |
they combine all three of the following advantages: |
they combine all three of the following advantages: |
|
|
@itemize |
@itemize @bullet |
|
|
@item Ease of implementation |
@item Ease of implementation |
|
|
Line 129 Vmgen makes it even easier to implement
|
Line 238 Vmgen makes it even easier to implement
|
techniques for building efficient interpreters. |
techniques for building efficient interpreters. |
|
|
@c ******************************************************************** |
@c ******************************************************************** |
|
@node Concepts, Invoking Vmgen, Why interpreters?, Top |
@chapter Concepts |
@chapter Concepts |
|
|
|
@menu |
|
* Front end and VM interpreter:: Modularizing an interpretive system |
|
* Data handling:: Stacks, registers, immediate arguments |
|
* Dispatch:: From one VM instruction to the next |
|
@end menu |
|
|
@c -------------------------------------------------------------------- |
@c -------------------------------------------------------------------- |
@section Front-end and virtual machine interpreter |
@node Front end and VM interpreter, Data handling, Concepts, Concepts |
|
@section Front end and VM interpreter |
|
|
@cindex front-end |
@cindex front-end |
Interpretive systems are typically divided into a @emph{front end} that |
Interpretive systems are typically divided into a @emph{front end} that |
Line 151 interpreter, except for VM branch instru
|
Line 268 interpreter, except for VM branch instru
|
structures. The conceptual similarity to real machine code results in |
structures. The conceptual similarity to real machine code results in |
the name @emph{virtual machine}. |
the name @emph{virtual machine}. |
|
|
In this framework, vmgen supports building the VM interpreter and any |
In this framework, Vmgen supports building the VM interpreter and any |
other component dealing with VM instructions. It does not have any |
other component dealing with VM instructions. It does not have any |
support for the front end, apart from VM code generation support. The |
support for the front end, apart from VM code generation support. The |
front end can be implemented with classical compiler front-end |
front end can be implemented with classical compiler front-end |
Line 164 Vmgen currently has no special support f
|
Line 281 Vmgen currently has no special support f
|
information in the instruction descriptions can be helpful, and we are |
information in the instruction descriptions can be helpful, and we are |
open for feature requests and suggestions. |
open for feature requests and suggestions. |
|
|
|
@c -------------------------------------------------------------------- |
|
@node Data handling, Dispatch, Front end and VM interpreter, Concepts |
@section Data handling |
@section Data handling |
|
|
@cindex stack machine |
@cindex stack machine |
Line 176 significantly more complex to implement
|
Line 295 significantly more complex to implement
|
Vmgen has special support and optimizations for stack VMs, making their |
Vmgen has special support and optimizations for stack VMs, making their |
implementation easy and efficient. |
implementation easy and efficient. |
|
|
You can also implement a register VM with vmgen (@pxref{Register |
You can also implement a register VM with Vmgen (@pxref{Register |
Machines}), and you will still profit from most vmgen features. |
Machines}), and you will still profit from most Vmgen features. |
|
|
@cindex stack item size |
@cindex stack item size |
@cindex size, stack items |
@cindex size, stack items |
Line 191 the data on the stack.
|
Line 310 the data on the stack.
|
@cindex immediate arguments |
@cindex immediate arguments |
Another source of data is immediate arguments VM instructions (in the VM |
Another source of data is immediate arguments VM instructions (in the VM |
instruction stream). The VM instruction stream is handled similar to a |
instruction stream). The VM instruction stream is handled similar to a |
stack in vmgen. |
stack in Vmgen. |
|
|
@cindex garbage collection |
@cindex garbage collection |
@cindex reference counting |
@cindex reference counting |
Line 202 harder, but might be possible (contact u
|
Line 321 harder, but might be possible (contact u
|
@c reference counting might be possible by including counting code in |
@c reference counting might be possible by including counting code in |
@c the conversion macros. |
@c the conversion macros. |
|
|
|
@c -------------------------------------------------------------------- |
|
@node Dispatch, , Data handling, Concepts |
@section Dispatch |
@section Dispatch |
|
|
Understanding this section is probably not necessary for using vmgen, |
Understanding this section is probably not necessary for using Vmgen, |
but it may help. You may want to skip it now, and read it if you find statements about dispatch methods confusing. |
but it may help. You may want to skip it now, and read it if you find statements about dispatch methods confusing. |
|
|
After executing one VM instruction, the VM interpreter has to dispatch |
After executing one VM instruction, the VM interpreter has to dispatch |
the next VM instruction (vmgen calls the dispatch routine @samp{NEXT}). |
the next VM instruction (Vmgen calls the dispatch routine @samp{NEXT}). |
Vmgen supports two methods of dispatch: |
Vmgen supports two methods of dispatch: |
|
|
@table |
@table @asis |
|
|
@item switch dispatch |
@item switch dispatch |
In this method the VM interpreter contains a giant @code{switch} |
In this method the VM interpreter contains a giant @code{switch} |
Line 229 Dispatch consists of loading this addres
|
Line 350 Dispatch consists of loading this addres
|
incrementing the VM instruction pointer. Typically the threaded-code |
incrementing the VM instruction pointer. Typically the threaded-code |
dispatch code is appended directly to the code for executing the VM |
dispatch code is appended directly to the code for executing the VM |
instruction. Threaded code cannot be implemented in ANSI C, but it can |
instruction. Threaded code cannot be implemented in ANSI C, but it can |
be implemented using GNU C's labels-as-values extension (@pxref{labels |
be implemented using GNU C's labels-as-values extension (@pxref{Labels |
as values}). |
as Values, , Labels as Values, gcc.info, GNU C Manual}). |
|
|
@end table |
@end table |
|
|
@c ************************************************************* |
@c ************************************************************* |
@chapter Invoking vmgen |
@node Invoking Vmgen, Example, Concepts, Top |
|
@chapter Invoking Vmgen |
|
|
The usual way to invoke vmgen is as follows: |
The usual way to invoke Vmgen is as follows: |
|
|
@example |
@example |
vmgen @var{infile} |
vmgen @var{infile} |
Line 251 current working directory) and replacing
|
Line 373 current working directory) and replacing
|
and @file{-peephole.i}. E.g., @command{bison hack/foo.vmg} will create |
and @file{-peephole.i}. E.g., @command{bison hack/foo.vmg} will create |
@file{foo-vm.i} etc. |
@file{foo-vm.i} etc. |
|
|
The command-line options supported by vmgen are |
The command-line options supported by Vmgen are |
|
|
@table @option |
@table @option |
|
|
Line 271 Print version and exit
|
Line 393 Print version and exit
|
@c env vars GFORTHDIR GFORTHDATADIR |
@c env vars GFORTHDIR GFORTHDATADIR |
|
|
@c **************************************************************** |
@c **************************************************************** |
|
@node Example, Input File Format, Invoking Vmgen, Top |
@chapter Example |
@chapter Example |
|
|
|
@menu |
|
* Example overview:: |
|
* Using profiling to create superinstructions:: |
|
@end menu |
|
|
|
@c -------------------------------------------------------------------- |
|
@node Example overview, Using profiling to create superinstructions, Example, Example |
@section Example overview |
@section Example overview |
|
|
There are two versions of the same example for using vmgen: |
There are two versions of the same example for using Vmgen: |
@file{vmgen-ex} and @file{vmgen-ex2} (you can also see Gforth as |
@file{vmgen-ex} and @file{vmgen-ex2} (you can also see Gforth as |
example, but it uses additional (undocumented) features, and also |
example, but it uses additional (undocumented) features, and also |
differs in some other respects). The example implements @emph{mini}, a |
differs in some other respects). The example implements @emph{mini}, a |
Line 320 stat.awk script for aggregatin
|
Line 450 stat.awk script for aggregatin
|
seq2rule.awk script for creating superinstructions |
seq2rule.awk script for creating superinstructions |
@end example |
@end example |
|
|
|
@noindent |
You would typically change much in or replace the following files: |
You would typically change much in or replace the following files: |
|
|
@example |
@example |
Line 342 check}. You can run run mini programs l
|
Line 473 check}. You can run run mini programs l
|
|
|
To learn about the options, type @samp{./mini -h}. |
To learn about the options, type @samp{./mini -h}. |
|
|
|
@c -------------------------------------------------------------------- |
|
@node Using profiling to create superinstructions, , Example overview, Example |
@section Using profiling to create superinstructions |
@section Using profiling to create superinstructions |
|
|
I have not added rules for this in the @file{Makefile} (there are many |
I have not added rules for this in the @file{Makefile} (there are many |
Line 392 preceed larger superinstructions.
|
Line 525 preceed larger superinstructions.
|
Now you can create a version of mini with superinstructions by just |
Now you can create a version of mini with superinstructions by just |
saying @samp{make} |
saying @samp{make} |
|
|
|
|
@c *************************************************************** |
@c *************************************************************** |
|
@node Input File Format, Using the generated code, Example, Top |
@chapter Input File Format |
@chapter Input File Format |
|
|
Vmgen takes as input a file containing specifications of virtual machine |
Vmgen takes as input a file containing specifications of virtual machine |
Line 400 instructions. This file usually has a n
|
Line 535 instructions. This file usually has a n
|
|
|
Most examples are taken from the example in @file{vmgen-ex}. |
Most examples are taken from the example in @file{vmgen-ex}. |
|
|
|
@menu |
|
* Input File Grammar:: |
|
* Simple instructions:: |
|
* Superinstructions:: |
|
* Register Machines:: How to define register VM instructions |
|
@end menu |
|
|
|
@c -------------------------------------------------------------------- |
|
@node Input File Grammar, Simple instructions, Input File Format, Input File Format |
@section Input File Grammar |
@section Input File Grammar |
|
|
The grammar is in EBNF format, with @code{@var{a}|@var{b}} meaning |
The grammar is in EBNF format, with @code{@var{a}|@var{b}} meaning |
Line 411 spaces and especially newlines; it's not
|
Line 555 spaces and especially newlines; it's not
|
any sequence of spaces and tabs is equivalent to a single space. |
any sequence of spaces and tabs is equivalent to a single space. |
|
|
@example |
@example |
description: {instruction|comment|eval-escape} |
description: @{instruction|comment|eval-escape@} |
|
|
instruction: simple-inst|superinst |
instruction: simple-inst|superinst |
|
|
simple-inst: ident " (" stack-effect " )" newline c-code newline newline |
simple-inst: ident " (" stack-effect " )" newline c-code newline newline |
|
|
stack-effect: {ident} " --" {ident} |
stack-effect: @{ident@} " --" @{ident@} |
|
|
super-inst: ident " =" ident {ident} |
super-inst: ident " =" ident @{ident@} |
|
|
comment: "\ " text newline |
comment: "\ " text newline |
|
|
Line 431 Note that the @code{\}s in this grammar
|
Line 575 Note that the @code{\}s in this grammar
|
C-style encodings for non-printable characters. |
C-style encodings for non-printable characters. |
|
|
The C code in @code{simple-inst} must not contain empty lines (because |
The C code in @code{simple-inst} must not contain empty lines (because |
vmgen would mistake that as the end of the simple-inst. The text in |
Vmgen would mistake that as the end of the simple-inst. The text in |
@code{comment} and @code{eval-escape} must not contain a newline. |
@code{comment} and @code{eval-escape} must not contain a newline. |
@code{Ident} must conform to the usual conventions of C identifiers |
@code{Ident} must conform to the usual conventions of C identifiers |
(otherwise the C compiler would choke on the vmgen output). |
(otherwise the C compiler would choke on the Vmgen output). |
|
|
Vmgen understands a few extensions beyond the grammar given here, but |
Vmgen understands a few extensions beyond the grammar given here, but |
these extensions are only useful for building Gforth. You can find a |
these extensions are only useful for building Gforth. You can find a |
description of the format used for Gforth in @file{prim}. |
description of the format used for Gforth in @file{prim}. |
|
|
@subsection |
@subsection Eval escapes |
@c woanders? |
@c woanders? |
The text in @code{eval-escape} is Forth code that is evaluated when |
The text in @code{eval-escape} is Forth code that is evaluated when |
vmgen reads the line. If you do not know (and do not want to learn) |
Vmgen reads the line. If you do not know (and do not want to learn) |
Forth, you can build the text according to the following grammar; these |
Forth, you can build the text according to the following grammar; these |
rules are normally all Forth you need for using vmgen: |
rules are normally all Forth you need for using Vmgen: |
|
|
@example |
@example |
text: stack-decl|type-prefix-decl|stack-prefix-decl |
text: stack-decl|type-prefix-decl|stack-prefix-decl |
Line 472 stack-prefix ( stack "prefix" -- )
|
Line 616 stack-prefix ( stack "prefix" -- )
|
@end example |
@end example |
|
|
|
|
|
@c -------------------------------------------------------------------- |
|
@node Simple instructions, Superinstructions, Input File Grammar, Input File Format |
@section Simple instructions |
@section Simple instructions |
|
|
We will use the following simple VM instruction description as example: |
We will use the following simple VM instruction description as example: |
Line 489 just plain C code.
|
Line 635 just plain C code.
|
The stack effect specifies that @code{sub} pulls two integers from the |
The stack effect specifies that @code{sub} pulls two integers from the |
data stack and puts them in the C variables @code{i1} and @code{i2} (with |
data stack and puts them in the C variables @code{i1} and @code{i2} (with |
the rightmost item (@code{i2}) taken from the top of stack) and later |
the rightmost item (@code{i2}) taken from the top of stack) and later |
pushes one integer (@code{i)) on the data stack (the rightmost item is |
pushes one integer (@code{i}) on the data stack (the rightmost item is |
on the top afterwards). |
on the top afterwards). |
|
|
How do we know the type and stack of the stack items? Vmgen uses |
How do we know the type and stack of the stack items? Vmgen uses |
Line 516 This line defines the stack @code{data-s
|
Line 662 This line defines the stack @code{data-s
|
pointer @code{sp}, and each item has the basic type @code{Cell}; other |
pointer @code{sp}, and each item has the basic type @code{Cell}; other |
types have to fit into one or two @code{Cell}s (depending on whether the |
types have to fit into one or two @code{Cell}s (depending on whether the |
type is @code{single} or @code{double} wide), and are converted from and |
type is @code{single} or @code{double} wide), and are converted from and |
to Cells on accessing the @code{data-stack) with conversion macros |
to Cells on accessing the @code{data-stack} with conversion macros |
(@pxref{Conversion macros}). Stacks grow towards lower addresses in |
(@pxref{VM engine}). Stacks grow towards lower addresses in |
vmgen-erated interpreters. |
Vmgen-erated interpreters. |
|
|
We can override the default stack of a stack item by using a stack |
We can override the default stack of a stack item by using a stack |
prefix. E.g., consider the following instruction: |
prefix. E.g., consider the following instruction: |
Line 548 arguments can only appear to the left of
|
Line 694 arguments can only appear to the left of
|
If there are multiple instruction stream arguments, the leftmost is the |
If there are multiple instruction stream arguments, the leftmost is the |
first one (just as the intuition suggests). |
first one (just as the intuition suggests). |
|
|
@subsubsection C Code Macros |
@menu |
|
* C Code Macros:: Macros recognized by Vmgen |
|
* C Code restrictions:: Vmgen makes assumptions about C code |
|
@end menu |
|
|
|
@c -------------------------------------------------------------------- |
|
@node C Code Macros, C Code restrictions, Simple instructions, Simple instructions |
|
@subsection C Code Macros |
|
|
Vmgen recognizes the following strings in the C code part of simple |
Vmgen recognizes the following strings in the C code part of simple |
instructions: |
instructions: |
Line 556 instructions:
|
Line 709 instructions:
|
@table @samp |
@table @samp |
|
|
@item SET_IP |
@item SET_IP |
As far as vmgen is concerned, a VM instruction containing this ends a VM |
As far as Vmgen is concerned, a VM instruction containing this ends a VM |
basic block (used in profiling to delimit profiled sequences). On the C |
basic block (used in profiling to delimit profiled sequences). On the C |
level, this also sets the instruction pointer. |
level, this also sets the instruction pointer. |
|
|
Line 571 middle of the C code, you need to use @s
|
Line 724 middle of the C code, you need to use @s
|
is a conditional VM branch: |
is a conditional VM branch: |
|
|
@example |
@example |
if (branch_condition) { |
if (branch_condition) @{ |
SET_IP(target); TAIL; |
SET_IP(target); TAIL; |
} |
@} |
/* implicit tail follows here */ |
/* implicit tail follows here */ |
@end example |
@end example |
|
|
Line 589 is not yet implemented in the vmgen-ex c
|
Line 742 is not yet implemented in the vmgen-ex c
|
typical application is in conditional VM branches: |
typical application is in conditional VM branches: |
|
|
@example |
@example |
if (branch_condition) { |
if (branch_condition) @{ |
SET_IP(target); TAIL; /* now this TAIL is necessary */ |
SET_IP(target); TAIL; /* now this TAIL is necessary */ |
} |
@} |
SUPER_CONTINUE; |
SUPER_CONTINUE; |
@end example |
@end example |
|
|
@end table |
@end table |
|
|
Note that vmgen is not smart about C-level tokenization, comments, |
Note that Vmgen is not smart about C-level tokenization, comments, |
strings, or conditional compilation, so it will interpret even a |
strings, or conditional compilation, so it will interpret even a |
commented-out SUPER_END as ending a basic block (or, e.g., |
commented-out SUPER_END as ending a basic block (or, e.g., |
@samp{RETAIL;} as @samp{TAIL;}). Conversely, vmgen requires the literal |
@samp{RETAIL;} as @samp{TAIL;}). Conversely, Vmgen requires the literal |
presence of these strings; vmgen will not see them if they are hiding in |
presence of these strings; Vmgen will not see them if they are hiding in |
a C preprocessor macro. |
a C preprocessor macro. |
|
|
|
|
@subsubsection C Code restrictions |
@c -------------------------------------------------------------------- |
|
@node C Code restrictions, , C Code Macros, Simple instructions |
|
@subsection C Code restrictions |
|
|
Vmgen generates code and performs some optimizations under the |
Vmgen generates code and performs some optimizations under the |
assumption that the user-supplied C code does not access the stack |
assumption that the user-supplied C code does not access the stack |
Line 617 the following.
|
Line 772 the following.
|
Accessing a stack or stack pointer directly can be a problem for several |
Accessing a stack or stack pointer directly can be a problem for several |
reasons: |
reasons: |
|
|
@itemize |
@itemize @bullet |
|
|
@item |
@item |
You may cache the top-of-stack item in a local variable (that is |
You may cache the top-of-stack item in a local variable (that is |
Line 633 automatically by mentioning a special st
|
Line 788 automatically by mentioning a special st
|
@c sometimes flushing and/or reloading unnecessary |
@c sometimes flushing and/or reloading unnecessary |
|
|
@item |
@item |
The vmgen-erated code loads the stack items from stack-pointer-indexed |
The Vmgen-erated code loads the stack items from stack-pointer-indexed |
memory into variables before the user-supplied C code, and stores them |
memory into variables before the user-supplied C code, and stores them |
from variables to stack-pointer-indexed memory afterwards. If you do |
from variables to stack-pointer-indexed memory afterwards. If you do |
any writes to the stack through its stack pointer in your C code, it |
any writes to the stack through its stack pointer in your C code, it |
Line 656 macros can be implemented in several way
|
Line 811 macros can be implemented in several way
|
contents. |
contents. |
|
|
|
|
|
@c -------------------------------------------------------------------- |
|
@node Superinstructions, Register Machines, Simple instructions, Input File Format |
@section Superinstructions |
@section Superinstructions |
|
|
|
Note: don't invest too much work in (static) superinstructions; a future |
|
version of Vmgen will support dynamic superinstructions (see Ian |
|
Piumarta and Fabio Riccardi, @cite{Optimizing Direct Threaded Code by |
|
Selective Inlining}, PLDI'98), and static superinstructions have much |
|
less benefit in that context. |
|
|
Here is an example of a superinstruction definition: |
Here is an example of a superinstruction definition: |
|
|
@example |
@example |
Line 668 lit_sub = lit sub
|
Line 831 lit_sub = lit sub
|
@code{sub} are its components. This superinstruction performs the same |
@code{sub} are its components. This superinstruction performs the same |
action as the sequence @code{lit} and @code{sub}. It is generated |
action as the sequence @code{lit} and @code{sub}. It is generated |
automatically by the VM code generation functions whenever that sequence |
automatically by the VM code generation functions whenever that sequence |
occurs, so you only need to add this definition if you want to use this |
occurs, so if you want to use this superinstruction, you just need to |
superinstruction (and even that can be partially automatized, |
add this definition (and even that can be partially automatized, |
@pxref{...}). |
@pxref{VM profiler}). |
|
|
Vmgen requires that the component instructions are simple instructions |
Vmgen requires that the component instructions are simple instructions |
defined before superinstructions using the components. Currently, vmgen |
defined before superinstructions using the components. Currently, Vmgen |
also requires that all the subsequences at the start of a |
also requires that all the subsequences at the start of a |
superinstruction (prefixes) must be defined as superinstruction before |
superinstruction (prefixes) must be defined as superinstruction before |
the superinstruction. I.e., if you want to define a superinstruction |
the superinstruction. I.e., if you want to define a superinstruction |
Line 695 sumof4 = add add add
|
Line 858 sumof4 = add add add
|
Here, @code{sumof4} is the longest prefix of @code{sumof5}, and @code{sumof3} |
Here, @code{sumof4} is the longest prefix of @code{sumof5}, and @code{sumof3} |
is the longest prefix of @code{sumof4}. |
is the longest prefix of @code{sumof4}. |
|
|
Note that vmgen assumes that only the code it generates accesses stack |
Note that Vmgen assumes that only the code it generates accesses stack |
pointers, the instruction pointer, and various stack items, and it |
pointers, the instruction pointer, and various stack items, and it |
performs optimizations based on this assumption. Therefore, VM |
performs optimizations based on this assumption. Therefore, VM |
instructions that change the instruction pointer should only be used as |
instructions that change the instruction pointer should only be used as |
Line 703 last component; a VM instruction that ac
|
Line 866 last component; a VM instruction that ac
|
not be used as component at all. Vmgen does not check these |
not be used as component at all. Vmgen does not check these |
restrictions, they just result in bugs in your interpreter. |
restrictions, they just result in bugs in your interpreter. |
|
|
|
@node Register Machines, , Superinstructions, Input File Format |
|
@section Register Machines |
|
|
|
If you want to implement a register VM rather than a stack VM with |
|
Vmgen, there are two ways to do it: Directly and through |
|
superinstructions. |
|
|
|
If you use the direct way, you define instructions that take the |
|
register numbers as immediate arguments, like this: |
|
|
|
@example |
|
add3 ( #src1 #src2 #dest -- ) |
|
reg[dest] = reg[src1]+reg[src2]; |
|
@end example |
|
|
|
If you use superinstructions to define a register VM, you define simple |
|
instructions that use a stack, and then define superinstructions that |
|
have no overall stack effect, like this: |
|
|
|
@example |
|
loadreg ( #src -- n ) |
|
n = reg[src]; |
|
|
|
storereg ( n #dest -- ) |
|
reg[dest] = n; |
|
|
|
adds ( n1 n2 -- n ) |
|
n = n1+n2; |
|
|
|
add3 = loadreg loadreg adds storereg |
|
@end example |
|
|
|
An advantage of this method is that you see the values and not just the |
|
register numbers in tracing (actually, with an appropriate definition of |
|
@code{printarg_src} (@pxref{VM engine}), you can print the values of the |
|
source registers on entry, but you cannot print the value of the |
|
destination register on exit. A disadvantage of this method is that |
|
currently you cannot generate superinstructions directly, but only |
|
through generating a sequence of simple instructions (we might change |
|
this in the future if there is demand). |
|
|
|
Could the register VM support be improved, apart from the issues |
|
mentioned above? It is hard to see how to do it in a general way, |
|
because there are a number of different designs that different people |
|
mean when they use the term @emph{register machine} in connection with |
|
VM interpreters. However, if you have ideas or requests in that |
|
direction, please let me know (@pxref{Contact}). |
|
|
@c ******************************************************************** |
@c ******************************************************************** |
|
@node Using the generated code, Changes, Input File Format, Top |
@chapter Using the generated code |
@chapter Using the generated code |
|
|
The easiest way to create a working VM interpreter with vmgen is |
The easiest way to create a working VM interpreter with Vmgen is |
probably to start with one of the examples, and modify it for your |
probably to start with one of the examples, and modify it for your |
purposes. This chapter is just the reference manual for the macros |
purposes. This chapter is just the reference manual for the macros |
etc. used by the generated code, and the other context expected by the |
etc. used by the generated code, the other context expected by the |
generated code, and what you can do with the various generated files. |
generated code, and what you can do with the various generated files. |
|
|
|
@menu |
|
* VM engine:: Executing VM code |
|
* VM instruction table:: |
|
* VM code generation:: Creating VM code (in the front-end) |
|
* Peephole optimization:: Creating VM superinstructions |
|
* VM disassembler:: for debugging the front end |
|
* VM profiler:: for finding worthwhile superinstructions |
|
@end menu |
|
|
|
@c -------------------------------------------------------------------- |
|
@node VM engine, VM instruction table, Using the generated code, Using the generated code |
@section VM engine |
@section VM engine |
|
|
The VM engine is the VM interpreter that executes the VM code. It is |
The VM engine is the VM interpreter that executes the VM code. It is |
Line 737 The following macros and variables are u
|
Line 959 The following macros and variables are u
|
|
|
@item LABEL(@var{inst_name}) |
@item LABEL(@var{inst_name}) |
This is used just before each VM instruction to provide a jump or |
This is used just before each VM instruction to provide a jump or |
@code{switch} label (the @samp{:} is provided by vmgen). For switch |
@code{switch} label (the @samp{:} is provided by Vmgen). For switch |
dispatch this should expand to @samp{case @var{label}}; for |
dispatch this should expand to @samp{case @var{label}}; for |
threaded-code dispatch this should just expand to @samp{case |
threaded-code dispatch this should just expand to @samp{case |
@var{label}}. In either case @var{label} is usually the @var{inst_name} |
@var{label}}. In either case @var{label} is usually the @var{inst_name} |
with some prefix or suffix to avoid naming conflicts. |
with some prefix or suffix to avoid naming conflicts. |
|
|
|
@item LABEL2(@var{inst_name}) |
|
This will be used for dynamic superinstructions; at the moment, this |
|
should expand to nothing. |
|
|
@item NAME(@var{inst_name_string}) |
@item NAME(@var{inst_name_string}) |
Called on entering a VM instruction with a string containing the name of |
Called on entering a VM instruction with a string containing the name of |
the VM instruction as parameter. In normal execution this should be a |
the VM instruction as parameter. In normal execution this should be a |
Line 770 macros do nothing. Then also related ma
|
Line 996 macros do nothing. Then also related ma
|
straightforward to define. For switch dispatch this code consists just |
straightforward to define. For switch dispatch this code consists just |
of a jump to the dispatch code (@samp{goto next_inst;} in our example; |
of a jump to the dispatch code (@samp{goto next_inst;} in our example; |
for direct threaded code it consists of something like |
for direct threaded code it consists of something like |
@samp{({cfa=*ip++; goto *cfa;})}. |
@samp{(@{cfa=*ip++; goto *cfa;@})}. |
|
|
Pulling code (usually the @samp{cfa=*ip;}) up into @samp{NEXT_P1} |
Pulling code (usually the @samp{cfa=*ip;}) up into @samp{NEXT_P1} |
usually does not cause problems, but pulling things up into |
usually does not cause problems, but pulling things up into |
Line 781 extreme variant is to pull code up even
|
Line 1007 extreme variant is to pull code up even
|
the previous VM instruction (prefetching, useful on PowerPCs). |
the previous VM instruction (prefetching, useful on PowerPCs). |
|
|
@item INC_IP(@var{n}) |
@item INC_IP(@var{n}) |
This increments IP by @var{n}. |
This increments @code{IP} by @var{n}. |
|
|
|
@item SET_IP(@var{target}) |
|
This sets @code{IP} to @var{target}. |
|
|
@item vm_@var{A}2@var{B}(a,b) |
@item vm_@var{A}2@var{B}(a,b) |
Type casting macro that assigns @samp{a} (of type @var{A}) to @samp{b} |
Type casting macro that assigns @samp{a} (of type @var{A}) to @samp{b} |
Line 831 Macro for executing @var{expr}, if top-o
|
Line 1060 Macro for executing @var{expr}, if top-o
|
top-of-stack caching for @var{stackpointer}; otherwise it should do |
top-of-stack caching for @var{stackpointer}; otherwise it should do |
nothing. |
nothing. |
|
|
|
@item SUPER_END |
|
This is used by the VM profiler (@pxref{VM profiler}); it should not do |
|
anything in normal operation, and call @code{vm_count_block(IP)} for |
|
profiling. |
|
|
|
@item SUPER_CONTINUE |
|
This is just a hint to Vmgen and does nothing at the C level. |
|
|
@item VM_DEBUG |
@item VM_DEBUG |
If this is defined, the tracing code will be compiled in (slower |
If this is defined, the tracing code will be compiled in (slower |
interpretation, but better debugging). Our example compiles two |
interpretation, but better debugging). Our example compiles two |
Line 857 basic type of the stack.
|
Line 1094 basic type of the stack.
|
@end table |
@end table |
|
|
|
|
@section{VM instruction table} |
@c -------------------------------------------------------------------- |
|
@node VM instruction table, VM code generation, VM engine, Using the generated code |
|
@section VM instruction table |
|
|
For threaded code we also need to produce a table containing the labels |
For threaded code we also need to produce a table containing the labels |
of all VM instructions. This is needed for VM code generation |
of all VM instructions. This is needed for VM code generation |
Line 882 For switch dispatch, we also need to def
|
Line 1121 For switch dispatch, we also need to def
|
used as case labels in an @code{enum}. |
used as case labels in an @code{enum}. |
|
|
For both purposes (VM instruction table, and enum), the file |
For both purposes (VM instruction table, and enum), the file |
@file{@var{name}-labels.i} is generated by vmgen. You have to define |
@file{@var{name}-labels.i} is generated by Vmgen. You have to define |
the following macro used in this file: |
the following macro used in this file: |
|
|
@table @samp |
@table @samp |
Line 892 For switch dispatch, this is just the na
|
Line 1131 For switch dispatch, this is just the na
|
name as used in @samp{LABEL(@var{inst_name})}), for both uses of |
name as used in @samp{LABEL(@var{inst_name})}), for both uses of |
@file{@var{name}-labels.i}. For threaded-code dispatch, this is the |
@file{@var{name}-labels.i}. For threaded-code dispatch, this is the |
address of the label defined in @samp{LABEL(@var{inst_name})}); the |
address of the label defined in @samp{LABEL(@var{inst_name})}); the |
address is taken with @samp{&&} (@pxref{labels-as-values}). |
address is taken with @samp{&&} (@pxref{Labels as Values, , Labels as |
|
Values, gcc.info, GNU C Manual}). |
|
|
@end table |
@end table |
|
|
|
|
|
@c -------------------------------------------------------------------- |
|
@node VM code generation, Peephole optimization, VM instruction table, Using the generated code |
@section VM code generation |
@section VM code generation |
|
|
Vmgen generates VM code generation functions in @file{@var{name}-gen.i} |
Vmgen generates VM code generation functions in @file{@var{name}-gen.i} |
that the front end can call to generate VM code. This is essential for |
that the front end can call to generate VM code. This is essential for |
an interpretive system. |
an interpretive system. |
|
|
For a VM instruction @samp{x ( #a b #c -- d )}, vmgen generates a |
For a VM instruction @samp{x ( #a b #c -- d )}, Vmgen generates a |
function with the prototype |
function with the prototype |
|
|
@example |
@example |
Line 951 every type that you use as immediate arg
|
Line 1193 every type that you use as immediate arg
|
In addition to using these functions to generate code, you should call |
In addition to using these functions to generate code, you should call |
@code{BB_BOUNDARY} at every basic block entry point if you ever want to |
@code{BB_BOUNDARY} at every basic block entry point if you ever want to |
use superinstructions (or if you want to use the profiling supported by |
use superinstructions (or if you want to use the profiling supported by |
vmgen; however, this is mainly useful for selecting superinstructions). |
Vmgen; however, this is mainly useful for selecting superinstructions). |
If you use @code{BB_BOUNDARY}, you should also define it (take a look at |
If you use @code{BB_BOUNDARY}, you should also define it (take a look at |
its definition in @file{vmgen-ex/mini.y}). |
its definition in @file{vmgen-ex/mini.y}). |
|
|
Line 962 superinstruction at the branch), and bec
|
Line 1204 superinstruction at the branch), and bec
|
themselves to the profiler. |
themselves to the profiler. |
|
|
|
|
|
@c -------------------------------------------------------------------- |
|
@node Peephole optimization, VM disassembler, VM code generation, Using the generated code |
@section Peephole optimization |
@section Peephole optimization |
|
|
You need peephole optimization only if you want to use |
You need peephole optimization only if you want to use |
Line 990 instruction to branch to), you have to c
|
Line 1234 instruction to branch to), you have to c
|
(@pxref{VM code generation}) at branch targets. |
(@pxref{VM code generation}) at branch targets. |
|
|
|
|
|
@c -------------------------------------------------------------------- |
|
@node VM disassembler, VM profiler, Peephole optimization, Using the generated code |
@section VM disassembler |
@section VM disassembler |
|
|
A VM code disassembler is optional for an interpretive system, but |
A VM code disassembler is optional for an interpretive system, but |
Line 1023 VM instruction table.
|
Line 1269 VM instruction table.
|
@end table |
@end table |
|
|
|
|
|
@c -------------------------------------------------------------------- |
|
@node VM profiler, , VM disassembler, Using the generated code |
|
@section VM profiler |
|
|
|
The VM profiler is designed for getting execution and occurence counts |
|
for VM instruction sequences, and these counts can then be used for |
|
selecting sequences as superinstructions. The VM profiler is probably |
|
not useful as profiling tool for the interpretive system. I.e., the VM |
|
profiler is useful for the developers, but not the users of the |
|
interpretive system. |
|
|
|
The output of the profiler is: for each basic block (executed at least |
|
once), it produces the dynamic execution count of that basic block and |
|
all its subsequences; e.g., |
|
|
|
@example |
|
9227465 lit storelocal |
|
9227465 storelocal branch |
|
9227465 lit storelocal branch |
|
@end example |
|
|
|
I.e., a basic block consisting of @samp{lit storelocal branch} is |
|
executed 9227465 times. |
|
|
|
This output can be combined in various ways. E.g., |
|
@file{vmgen/stat.awk} adds up the occurences of a given sequence wrt |
|
dynamic execution, static occurence, and per-program occurence. E.g., |
|
|
|
@example |
|
2 16 36910041 loadlocal lit |
|
@end example |
|
|
|
indicates that the sequence @samp{loadlocal lit} occurs in 2 programs, |
|
in 16 places, and has been executed 36910041 times. Now you can select |
|
superinstructions in any way you like (note that compile time and space |
|
typically limit the number of superinstructions to 100--1000). After |
|
you have done that, @file{vmgen/seq2rule.awk} turns lines of the form |
|
above into rules for inclusion in a Vmgen input file. Note that this |
|
script does not ensure that all prefixes are defined, so you have to do |
|
that in other ways. So, an overall script for turning profiles into |
|
superinstructions can look like this: |
|
|
|
@example |
|
awk -f stat.awk fib.prof test.prof| |
|
awk '$3>=10000'| #select sequences |
|
fgrep -v -f peephole-blacklist| #eliminate wrong instructions |
|
awk -f seq2rule.awk| #turn into superinstructions |
|
sort -k 3 >mini-super.vmg #sort sequences |
|
@end example |
|
|
|
Here the dynamic count is used for selecting sequences (preliminary |
|
results indicate that the static count gives better results, though); |
|
the third line eliminats sequences containing instructions that must not |
|
occur in a superinstruction, because they access a stack directly. The |
|
dynamic count selection ensures that all subsequences (including |
|
prefixes) of longer sequences occur (because subsequences have at least |
|
the same count as the longer sequences); the sort in the last line |
|
ensures that longer superinstructions occur after their prefixes. |
|
|
|
But before using it, you have to have the profiler. Vmgen supports its |
|
creation by generating @file{@var{file}-profile.i}; you also need the |
|
wrapper file @file{vmgen-ex/profile.c} that you can use almost verbatim. |
|
|
|
The profiler works by recording the targets of all VM control flow |
|
changes (through @code{SUPER_END} during execution, and through |
|
@code{BB_BOUNDARY} in the front end), and counting (through |
|
@code{SUPER_END}) how often they were targeted. After the program run, |
|
the numbers are corrected such that each VM basic block has the correct |
|
count (originally entering a block without executing a branch does not |
|
increase the count), then the subsequences of all basic blocks are |
|
printed. To get all this, you just have to define @code{SUPER_END} (and |
|
@code{BB_BOUNDARY}) appropriately, and call @code{vm_print_profile(FILE |
|
*file)} when you want to output the profile on @code{file}. |
|
|
|
The @file{@var{file}-profile.i} is simular to the disassembler file, and |
|
it uses variables and functions defined in @file{vmgen-ex/profile.c}, |
|
plus @code{VM_IS_INST} already defined for the VM disassembler |
|
(@pxref{VM disassembler}). |
|
|
|
|
|
@c ********************************************************** |
|
@node Changes, Contact, Using the generated code, Top |
|
@chapter Changes |
|
|
|
Users of the gforth-0.5.9-20010501 version of Vmgen need to change |
|
several things in their source code to use the current version. I |
|
recommend keeping the gforth-0.5.9-20010501 version until you have |
|
completed the change (note that you can have several versions of Gforth |
|
installed at the same time). I hope to avoid such incompatible changes |
|
in the future. |
|
|
|
The required changes are: |
|
|
|
@table @code |
|
|
|
@item vm_@var{A}2@var{B} |
|
now takes two arguments. |
|
|
|
@item vm_two@var{A}2@var{B}(b,a1,a2); |
|
changed to vm_two@var{A}2@var{B}(a1,a2,b) (note the absence of the @samp{;}). |
|
|
|
@end table |
|
|
|
Also some new macros have to be defined, e.g., @code{INST_ADDR}, and |
|
@code{LABEL}; some macros have to be defined in new contexts, e.g., |
|
@code{VM_IS_INST} is now also needed in the disassembler. |
|
|
|
@node Contact, Copying This Manual, Changes, Top |
|
@chapter Contact |
|
|
|
@node Copying This Manual, Index, Contact, Top |
|
@appendix Copying This Manual |
|
|
Invocation |
@menu |
|
* GNU Free Documentation License:: License for copying this manual. |
|
@end menu |
|
|
Input Syntax |
@include fdl.texi |
|
|
Concepts: Front end, VM, Stacks, Types, input stream |
|
|
|
Contact |
@node Index, , Copying This Manual, Top |
|
@unnumbered Index |
|
|
|
@printindex cp |
|
|
Required changes: |
@bye |
vm_...2... -> two arguments |
|
"vm_two...2...(arg1,arg2,arg3);" -> "vm_two...2...(arg3,arg1,arg2)" (no ";"). |
|
define INST_ADDR and LABEL |
|
define VM_IS_INST also for disassembler |
|