Return to libcc.fs CVS log

Up to [gforth] / gforth

libcc.fs now uses libtool
OPEN-LIB and LIB-SYM now call libltdl (libtool) functions
  probably needs some additional adjustment in configure.in
added LIB-ERROR

    1: \ libcc.fs	foreign function interface implemented using a C compiler
    2: 
    3: \ Copyright (C) 2006 Free Software Foundation, Inc.
    4: 
    5: \ This file is part of Gforth.
    6: 
    7: \ Gforth is free software; you can redistribute it and/or
    8: \ modify it under the terms of the GNU General Public License
    9: \ as published by the Free Software Foundation; either version 2
   10: \ of the License, or (at your option) any later version.
   11: 
   12: \ This program is distributed in the hope that it will be useful,
   13: \ but WITHOUT ANY WARRANTY; without even the implied warranty of
   14: \ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   15: \ GNU General Public License for more details.
   16: 
   17: \ You should have received a copy of the GNU General Public License
   18: \ along with this program; if not, write to the Free Software
   19: \ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
   20: 
   21: 
   22: \ What this implementation does is this: if it sees a declaration like
   23: 
   24: \ \ something that tells it that the current library is libc
   25: \ \c #include <unistd.h>
   26: \ c-function dlseek lseek n d n -- d
   27: 
   28: \ it genererates C code similar to the following:
   29: 
   30: \ #include <gforth.h>
   31: \ #include <unistd.h>
   32: \ 
   33: \ void gforth_c_lseek_ndn_d(void)
   34: \ {
   35: \   Cell *sp = gforth_SP;
   36: \   Float *fp = gforth_FP;
   37: \   long long result;  /* longest type in C */
   38: \   gforth_ll2d(lseek(sp[3],gforth_d2ll(sp[2],sp[1]),sp[0]),sp[3],sp[2]);
   39: \   gforth_SP = sp+2;
   40: \ }
   41: 
   42: \ Then it compiles this code and dynamically links it into the Gforth
   43: \ system (batching and caching are future work).  It also dynamically
   44: \ links lseek.  Performing DLSEEK then puts the function pointer of
   45: \ the function pointer of gforth_c_lseek_ndn_d on the stack and
   46: \ calls CALL-C.
   47: 
   48: \ ToDo:
   49: 
   50: \ Batching, caching and lazy evaluation:
   51: 
   52: \ Batching:
   53: 
   54: \ New words are deferred, and the corresponding C functions are
   55: \ collected in one file, until the first word is EXECUTEd; then the
   56: \ file is compiled and linked into the system, and the word is
   57: \ resolved.
   58: 
   59: \ Caching:
   60: 
   61: \ Instead of compiling all this stuff anew for every execution, we
   62: \ keep the files around and have an index file containing the function
   63: \ names and their corresponding .so files.  If the needed wrapper name
   64: \ is already present, it is just linked instead of generating the
   65: \ wrapper again.  This is all done by loading the index file(s?),
   66: \ which define words for the wrappers in a separate wordlist.
   67: 
   68: \ The files are built in .../lib/gforth/$VERSION/libcc/ or
   69: \ ~/.gforth/libcc/$HOST/.
   70: 
   71: \ other things to do:
   72: 
   73: \ c-variable forth-name c-name
   74: \ c-constant forth-name c-name
   75: 
   76: \ Todo: conversion between function pointers and xts (both directions)
   77: 
   78: \ taking an xt and turning it into a function pointer:
   79: 
   80: \ e.g., assume we have the xt of + and want to create a C function int
   81: \ gforth_callback_plus(int, int), and then pass the pointer to that
   82: \ function:
   83: 
   84: \ There should be Forth code like this:
   85: \   ] + 0 (bye)
   86: \ Assume that the start of this code is START
   87:         
   88: \ Now, there should be a C function:
   89: 
   90: \ int gforth_callback_plus(int p1, int p2)
   91: \ {
   92: \   Cell   *sp = gforth_SP;
   93: \   Float  *fp = gforth_FP;
   94: \   Float  *fp = gforth_FP;
   95: \   Address lp = gforth_LP;
   96: \   sp -= 2;
   97: \   sp[0] = p1;
   98: \   sp[1] = p2;
   99: \   gforth_engine(START, sp, rp, fp, lp);
  100: \   sp += 1;
  101: \   gforth_RP = rp;
  102: \   gforth_SP = sp;
  103: \   gforth_FP = fp;
  104: \   gforth_LP = lp;
  105: \   return sp[0];
  106: \ }
  107: 
  108: \ and the pointer to that function is the C function pointer for the XT of +.
  109: 
  110: \ Future problems:
  111: \   how to combine the Forth code generation with inlining
  112: \   START is not a constant across executions (when caching the C files)
  113: \      Solution: make START a variable, and store into it on startup with dlsym
  114: 
  115: \ Syntax:
  116: \  callback <rettype> <params> <paramtypes> -- <rettype>
  117: 
  118: 
  119: \ data structures
  120: 
  121: \ For every c-function, we have three words: two anonymous words
  122: \ created by c-function-ft (first time) and c-function-rt (run-time),
  123: \ and a named deferred word.  The deferred word first points to the
  124: \ first-time word, then to the run-time word; the run-time word calls
  125: \ the c function.
  126: 
  127: : delete-file 2drop 0 ;
  128: 
  129: require struct.fs
  130: 
  131:     \ counted-string
  132:     
  133: \ c-function-ft word body:
  134: struct
  135:     cell% field cff-cfr \ xt of c-function-rt word
  136:     cell% field cff-deferred \ xt of c-function deferred word
  137:     cell% field cff-lha \ address of the lib-handle for the lib that
  138:                         \ contains the wrapper function of the word
  139:     char% field cff-rtype  \ return type
  140:     char% field cff-np     \ number of parameters
  141:     1 0   field cff-ptypes \ #npar parameter types
  142:     \  counted string: c-name
  143: end-struct cff%
  144: 
  145: variable c-source-file-id \ contains the source file id of the current batch
  146: 0 c-source-file-id !
  147: variable lib-handle-addr \ points to the library handle of the current batch.
  148:                          \ the library handle is 0 if the current
  149:                          \ batch is not yet compiled.
  150: 2variable lib-filename   \ filename without extension
  151: 2variable lib-modulename \ basename of the file without extension
  152: 
  153: : .nb ( n -- )
  154:     0 .r ;
  155: 
  156: : const+ ( n1 "name" -- n2 )
  157:     dup constant 1+ ;
  158: 
  159: : front-string { c-addr1 u1 c-addr2 u2 -- c-addr3 u3 }
  160:     \ insert string c-addr2 u2 in buffer c-addr1 u1; c-addr3 u3 is the
  161:     \ remainder of the buffer.
  162:     assert( u1 u2 u>= )
  163:     c-addr2 c-addr1 u2 move
  164:     c-addr1 u1 u2 /string ;
  165: 
  166: : front-char { c-addr1 u1 c -- c-addr3 u2 }
  167:     \ insert c in buffer c-addr1 u1; c-addr3 u3 is the remainder of
  168:     \ the buffer.
  169:     assert( u1 0 u> )
  170:     c c-addr1 c!
  171:     c-addr1 u1 1 /string ;
  172: 
  173: : s+ { addr1 u1 addr2 u2 -- addr u }
  174:     u1 u2 + allocate throw { addr }
  175:     addr1 addr u1 move
  176:     addr2 addr u1 + u2 move
  177:     addr u1 u2 +
  178: ;
  179: 
  180: : append { addr1 u1 addr2 u2 -- addr u }
  181:     addr1 u1 u2 + dup { u } resize throw { addr }
  182:     addr2 addr u1 + u2 move
  183:     addr u ;
  184: 
  185: \ linked list stuff (should go elsewhere)
  186: 
  187: struct
  188:     cell% field list-next
  189:     1 0   field list-payload
  190: end-struct list%
  191: 
  192: : list-insert { node list -- }
  193:     list list-next @ node list-next !
  194:     node list list-next ! ;
  195: 
  196: : list-append { node endlistp -- }
  197:     \ insert node at place pointed to by endlistp
  198:     node endlistp @ list-insert
  199:     node list-next endlistp ! ;
  200: 
  201: : list-map ( ... list xt -- ... )
  202:     \ xt ( ... node -- ... )
  203:     { xt } begin { node }
  204: 	node while
  205: 	    node xt execute
  206: 	    node list-next @
  207:     repeat ;
  208: 
  209: \ C prefix lines
  210: 
  211: \ linked list of longcstrings: [ link | count-cell | characters ]
  212: 
  213: list%
  214:     cell% field c-prefix-count
  215:     1 0   field c-prefix-chars
  216: end-struct c-prefix%
  217: 
  218: variable c-prefix-lines 0 c-prefix-lines !
  219: variable c-prefix-lines-end c-prefix-lines c-prefix-lines-end !
  220: 
  221: : print-c-prefix-line ( node -- )
  222:     dup c-prefix-chars swap c-prefix-count @ type cr ;
  223: 
  224: : print-c-prefix-lines ( -- )
  225:     c-prefix-lines @ ['] print-c-prefix-line list-map ;
  226: 
  227: : save-c-prefix-line ( c-addr u -- )
  228:     c-source-file-id @ ?dup-if
  229: 	>r 2dup r> write-line throw
  230:     then
  231:     align here 0 , c-prefix-lines-end list-append ( c-addr u )
  232:     longstring, ;
  233: 
  234: : \c ( "rest-of-line" -- ) \ gforth backslash-c
  235:     \G One line of C declarations for the C interface
  236:     -1 parse save-c-prefix-line ;
  237: 
  238: s" #include <gforth/" version-string s+ s" /libcc.h>" append ( c-addr u )
  239:   2dup save-c-prefix-line drop free throw
  240: 
  241: \ Types (for parsing)
  242: 
  243: wordlist constant libcc-types
  244: 
  245: get-current libcc-types set-current
  246: 
  247: \ index values
  248: -1
  249: const+ -- \ end of arguments
  250: const+ n \ integer cell
  251: const+ a \ address cell
  252: const+ d \ double
  253: const+ r \ float
  254: const+ func \ C function pointer
  255: const+ void
  256: drop
  257: 
  258: set-current
  259: 
  260: : parse-libcc-type ( "libcc-type" -- u )
  261:     parse-name libcc-types search-wordlist 0= -13 and throw execute ;
  262: 
  263: : parse-function-types ( "{libcc-type}" "--" "libcc-type" -- )
  264:     here 2 chars allot here begin
  265: 	parse-libcc-type dup 0>= while
  266: 	    c,
  267:     repeat
  268:     drop here swap - over char+ c!
  269:     parse-libcc-type dup 0< -32 and throw swap c! ;
  270: 
  271: : type-letter ( n -- c )
  272:     chars s" nadrfv" drop + c@ ;
  273: 
  274: \ count-stacks
  275: 
  276: : count-stacks-n ( fp-change1 sp-change1 -- fp-change2 sp-change2 )
  277:     1+ ;
  278: 
  279: : count-stacks-a ( fp-change1 sp-change1 -- fp-change2 sp-change2 )
  280:     1+ ;
  281: 
  282: : count-stacks-d ( fp-change1 sp-change1 -- fp-change2 sp-change2 )
  283:     2 + ;
  284: 
  285: : count-stacks-r ( fp-change1 sp-change1 -- fp-change2 sp-change2 )
  286:     swap 1+ swap ;
  287: 
  288: : count-stacks-func ( fp-change1 sp-change1 -- fp-change2 sp-change2 )
  289:     1+ ;
  290: 
  291: : count-stacks-void ( fp-change1 sp-change1 -- fp-change2 sp-change2 )
  292: ;
  293: 
  294: create count-stacks-types
  295: ' count-stacks-n ,
  296: ' count-stacks-a ,
  297: ' count-stacks-d ,
  298: ' count-stacks-r ,
  299: ' count-stacks-func ,
  300: ' count-stacks-void ,
  301: 
  302: : count-stacks ( pars -- fp-change sp-change )
  303:     \ pars is an addr u pair
  304:     0 0 2swap over + swap u+do
  305: 	i c@ cells count-stacks-types + @ execute
  306:     loop ;
  307: 
  308: \ gen-pars
  309: 
  310: : gen-par-n ( fp-depth1 sp-depth1 -- fp-depth2 sp-depth2 )
  311:     ." sp[" 1- dup .nb ." ]" ;
  312: 
  313: : gen-par-a ( fp-depth1 sp-depth1 -- fp-depth2 sp-depth2 )
  314:     ." (void *)(" gen-par-n ." )" ;
  315: 
  316: : gen-par-d ( fp-depth1 sp-depth1 -- fp-depth2 sp-depth2 )
  317:     ." gforth_d2ll(" gen-par-n ." ," gen-par-n ." )" ;
  318: 
  319: : gen-par-r ( fp-depth1 sp-depth1 -- fp-depth2 sp-depth2 )
  320:     swap 1- tuck ." fp[" .nb ." ]" ;
  321: 
  322: : gen-par-func ( fp-depth1 sp-depth1 -- fp-depth2 sp-depth2 )
  323:     gen-par-a ;
  324: 
  325: : gen-par-void ( fp-depth1 sp-depth1 -- fp-depth2 sp-depth2 )
  326:     -32 throw ;
  327: 
  328: create gen-par-types
  329: ' gen-par-n ,
  330: ' gen-par-a ,
  331: ' gen-par-d ,
  332: ' gen-par-r ,
  333: ' gen-par-func ,
  334: ' gen-par-void ,
  335: 
  336: : gen-par ( fp-depth1 sp-depth1 partype -- fp-depth2 sp-depth2 )
  337:     cells gen-par-types + @ execute ;
  338: 
  339: \ the call itself
  340: 
  341: : gen-wrapped-call { d: pars d: c-name fp-change1 sp-change1 -- }
  342:     c-name type ." ("
  343:     fp-change1 sp-change1 pars over + swap u+do 
  344: 	i c@ gen-par
  345: 	i 1+ i' < if
  346: 	    ." ,"
  347: 	endif
  348:     loop
  349:     2drop ." )" ;
  350: 
  351: \ calls for various kinds of return values
  352: 
  353: : gen-wrapped-void ( pars c-name fp-change1 sp-change1 -- fp-change sp-change )
  354:     2dup 2>r gen-wrapped-call 2r> ;
  355: 
  356: : gen-wrapped-n ( pars c-name fp-change1 sp-change1 -- fp-change sp-change )
  357:     2dup gen-par-n 2>r ." =" gen-wrapped-call 2r> ;
  358: 
  359: : gen-wrapped-a ( pars c-name fp-change1 sp-change1 -- fp-change sp-change )
  360:     2dup gen-par-n 2>r ." =(Cell)" gen-wrapped-call 2r> ;
  361: 
  362: : gen-wrapped-d ( pars c-name fp-change1 sp-change1 -- fp-change sp-change )
  363:     ." gforth_ll2d(" gen-wrapped-void
  364:     ." ," gen-par-n ." ," gen-par-n ." )" ;
  365: 
  366: : gen-wrapped-r ( pars c-name fp-change1 sp-change1 -- fp-change sp-change )
  367:     2dup gen-par-r 2>r ." =" gen-wrapped-void 2r> ;
  368: 
  369: : gen-wrapped-func ( pars c-name fp-change1 sp-change1 -- fp-change sp-change )
  370:     gen-wrapped-a ;
  371: 
  372: create gen-wrapped-types
  373: ' gen-wrapped-n ,
  374: ' gen-wrapped-a ,
  375: ' gen-wrapped-d ,
  376: ' gen-wrapped-r ,
  377: ' gen-wrapped-func ,
  378: ' gen-wrapped-void ,
  379: 
  380: : gen-wrapped-stmt ( pars c-name fp-change1 sp-change1 ret -- fp-change sp-change )
  381:     cells gen-wrapped-types + @ execute ;
  382: 
  383: : wrapper-function-name ( addr -- c-addr u )
  384:     \ addr points to the return type index of a c-function descriptor
  385:     count { r-type } count { d: pars }
  386:     pars + count { d: c-name }
  387:     s" gforth_c_" { d: prefix }
  388:     prefix nip c-name nip + pars nip + 3 + { u }
  389:     u allocate throw { c-addr }
  390:     c-addr u
  391:     prefix front-string c-name front-string '_ front-char
  392:     pars bounds u+do
  393: 	i c@ type-letter front-char
  394:     loop
  395:     '_ front-char r-type type-letter front-char assert( dup 0= )
  396:     2drop c-addr u ;
  397: 
  398: : gen-wrapper-function ( addr -- )
  399:     \ addr points to the return type index of a c-function descriptor
  400:     dup { descriptor }
  401:     count { ret } count 2dup { d: pars } chars + count { d: c-name }
  402:     ." void " lib-modulename 2@ type ." _LTX_" descriptor wrapper-function-name 2dup type drop free throw
  403:     .\" (void)\n"
  404:     .\" {\n  Cell MAYBE_UNUSED *sp = gforth_SP;\n  Float MAYBE_UNUSED *fp = gforth_FP;\n  "
  405:     pars c-name 2over count-stacks ret gen-wrapped-stmt .\" ;\n"
  406:     ?dup-if
  407: 	."   gforth_SP = sp+" .nb .\" ;\n"
  408:     endif
  409:     ?dup-if
  410: 	."   gforth_FP = fp+" .nb .\" ;\n"
  411:     endif
  412:     .\" }\n" ;
  413: 
  414: : tempdir ( -- c-addr u )
  415:     s" TMPDIR" getenv dup 0= if
  416:         2drop s" /tmp"
  417:     then ;
  418: 
  419: : gen-filename ( x -- c-addr u )
  420:     \ generates a filename without extension for lib-handle-addr X
  421:     0 <<# ['] #s $10 base-execute #> 
  422:     tempdir s" /gforth_c_" s+ 2swap append #>> ;
  423: 
  424: : init-c-source-file ( -- )
  425:     c-source-file-id @ 0= if
  426:         here 0 , dup lib-handle-addr ! gen-filename 2dup lib-filename 2!
  427:         2dup tempdir nip 1+ /string lib-modulename 2!
  428: 	s" .c" s+ 2dup w/o create-file throw dup c-source-file-id !
  429:         ['] print-c-prefix-lines swap outfile-execute
  430:         drop free throw
  431:     endif ;
  432: 
  433: : c-source-file ( -- file-id )
  434:     c-source-file-id @ assert( dup ) ;
  435: 
  436: \ libtool --mode=compile gcc -I /nfs/a5/anton/gforth-amd64/include -O -c /tmp/gforth_c_2AAAAB2E7C50.c -o /tmp/gforth_c_2AAAAB2E7C50.lo
  437: \ libtool --mode=link gcc -module -rpath /tmp /tmp/gforth_c_2AAAAB2E7C50.lo -o /tmp/gforth_c_2AAAAB2E7C50.la
  438: 
  439: DEFER compile-wrapper-function
  440: :NONAME ( -- )
  441:     c-source-file close-file throw
  442:     0 c-source-file-id !
  443:     [ s" libtool --silent --mode=compile gcc -I "
  444:       s" includedir" getenv append ] sliteral
  445:     s"  -O -c " s+ lib-filename 2@ append s" .c -o " append
  446:     lib-filename 2@ append s" .lo" append ( c-addr u )
  447:     2dup system drop free throw $? abort" libtool compile failed"
  448:     s" libtool --silent --mode=link gcc -module -rpath " tempdir s+ s"  " append
  449:     lib-filename 2@ append s" .lo -o " append
  450:     lib-filename 2@ append s" .la" append ( c-addr u )
  451:     2dup system drop free throw $? abort" libtool link failed"
  452:     lib-filename 2@ s" .la" s+
  453:     2dup open-lib dup 0= if
  454:         cr lib-error type true abort" open-lib failed"
  455:     endif
  456:     ( lib-handle ) lib-handle-addr @ !
  457:     2dup delete-file throw drop free throw
  458:     lib-filename 2@ s" .c" s+ 2dup delete-file throw drop free throw
  459:     lib-filename 2@ drop free throw 0 0 lib-filename 2! ; IS compile-wrapper-function
  460: \    s" ar rcs xxx.a xxx.o" system
  461: \    $? abort" ar generated error" ;
  462: 
  463: : link-wrapper-function { cff -- sym }
  464:     cff cff-rtype wrapper-function-name { d: wrapper-name }
  465:     wrapper-name cff cff-lha @ @ assert( dup ) lib-sym dup 0= if
  466:         cr lib-error type -&32 throw
  467:     endif
  468:     wrapper-name drop free throw ;
  469: 
  470: : c-function-ft ( xt-defr xt-cfr "c-name" "{libcc-type}" "--" "libcc-type" -- )
  471:     \ build time/first time action for c-function
  472:     init-c-source-file
  473:     noname create 2, lib-handle-addr @ ,
  474:     parse-name { d: c-name }
  475:     here parse-function-types c-name string,
  476:     ['] gen-wrapper-function c-source-file outfile-execute
  477:   does> ( ... -- ... )
  478:     dup 2@ { xt-defer xt-cfr }
  479:     dup cff-lha @ @ 0= if
  480: 	compile-wrapper-function
  481:     endif
  482:     link-wrapper-function xt-cfr >body !
  483:     xt-cfr xt-defer defer!
  484:     xt-cfr execute ;
  485: 
  486: : c-function-rt ( -- )
  487:     \ run-time definition for c function; addr is the address where
  488:     \ the sym should be stored
  489:     noname create 0 ,
  490:   does> ( ... -- ... )
  491:     @ call-c ;
  492: 
  493: : c-function ( "forth-name" "c-name" "@{type@}" "--" "type" -- ) \ gforth
  494:     \G Define a Forth word @i{forth-name}.  @i{Forth-name} has the
  495:     \G specified stack effect and calls the C function @code{c-name}.
  496:     defer lastxt dup c-function-rt lastxt c-function-ft
  497:     lastxt swap defer! ;