Integrating FastCGI with Perl-5

Copyright © 1996 Open Market, Inc. 245 First Street, Cambridge, MA 02142 U.S.A.
Tel: 617-949-7000 URL: http://www.openmarket.com/
$Id: fcgi-perl.htm,v 1.5 1996/11/09 00:09:22 gambarin Exp $

• 1. Introduction
• 2. Perl with sfio and an FCGI module
• 3. Perl with fcgi_stdio and an FCGI module
  • 3.1 Basic recipe
  • 3.2 Semi-advanced recipe
  • 3.3 Advanced recipe
• 4. Writing FastCGI applications in Perl

1. Introduction

FastCGI has been integrated with Perl in two different ways:

1. By writing a module that plugs into any Perl interpreter that's been built with sfio, a stdio alternative from AT&T.
2. By writing a module that plugs into any Perl interpreter that's been built with FastCGI's fcgi_stdio library layering over stdio. The first approach, implemented by Sven Verdoolaege (skimo@breughel.ufsia.ac.be), is probably the better of the two, since sfio is a generally useful addition to Perl. The second approach, implemented by engineers at Open Market, predates the availability of an sfio-integrated Perl and demonstrates that the fcgi_stdio library can be used with a substantial C application.

   The two approaches are compatible at the Perl source code level; a Perl application developed using one approach can be run using the other. And both approaches result in a general-purpose Perl interpreter, not a Perl interpreter that's only usable for FastCGI applications.

   This memo documents both approaches and explains a small Perl FastCGI application.

   2. Perl with sfio and an FCGI module

   As of release 5 patch 3 subpatch 2 (5.003.02), Perl has announced an optional support for sfio (safe/fast string/file I/O), which is an alternative to stdio that AT&T distributes freely. An advantage of sfio over stdio is that sfio provides the ability to implement new stream classes that don't simply transfer sequential bytes to or from a file descriptor. This flexibility is exactly what FastCGI needs in order to implement the standard I/O streams in a manner that's transparent to applications.

   Perl interpreters incorporating sfio are not widely available in binary form, so most likely you'll have to build your own. Your build should go smoothly if you follow the instructions below. The instructions assume:

   • You are building Perl 5.0 patch level 3 subpatch level 2 (5.003.02) or higher. That's the first Perl release to support sfio.

   Follow these steps to build a Perl with sfio:

   1. Obtain sfio source code from ftp://ftp.funet.fi/pub/languages/perl/CPAN/src/misc

   2. Unpack the tar file using tar xvf command. $sfio will be used as a shorthand for the directory in which sfio package is installed.

   3. Update your $PATH variable as specified in $sfio/README and run make command in the $sfio/src/lib/sfio subdirectory.

   4. Rename or delete the file $sfio/include/stdio.h, since it may interfere in the further build process.

   5. Obtain Perl source (version 5 subversion 003 patchlevel 2 or higher) from http://fohnix.metronet.com/perlinfo/source/5.0/unsupported

   6. Unpack the tar file using tar xvf command. $perl is used as a shorthand for the directory that is created.

   7. Configure, build, and install Perl as follows:

      % cd $perl
      % ./Configure -Duseperlio -Dusesfio
      % make 
      % make test
      % make install
      

      There are certain Configure questions that must be answered differently when building Perl with sfio:

      Perl5 can now use alternate file IO mechanisms to ANSI stdio. However these are experimental and may cause problems with some extension modules. Use stdio as with previous versions? [y]

      You should answer no.

      Any additional cc flags?

      You should use the following cc flags along with any defaults that Perl Configure supplied:

      • -I$sfio/include

      Any additional ld flags (NOT including libraries):

      You should specify the following ld flags:

      • -L$sfio/lib

      Additional Libraries:

      Check that -lsfio is one of the specified libraries. Press return key to continue.

      NOTE: If you did not install Perl as a root user, make sure to correctly set environment variable PERL5LIB to indicate the location of Perl libraries. For example, if you installed Perl binary into the $INSTALL subdirectory and you are running Solaris, the following will set your proper library path:

      % setenv PERL5LIB $INSTALL/lib:$INSTALL/lib/sun4-solaris/perl5.003_02
      

   8. Obtain Perl/Sfio module for FastCGI support from ftp://ftp.funet.fi/pub/languages/perl/CPAN/authors/id/SKIMO

   9. Unpack FCGI module using tar command. We use $sfiomod to denote the subdirectory that is created in the process.

   10. Build and install the module with the following commands:

       % cd $sfiomod
       % $INSTALL/bin/perl Makefile.PL
       % make
       % make test
       % make install
       

   3. Perl with fcgi_stdio and an FCGI module

   3.1 Basic recipe

   Here are the assumptions embedded in the following recipe:
   • You are building Perl 5.0 Patch 2 (5.002) or higher, since all examples that are provided are based on that release.

   • You have gcc version 2.7 installed on your system, and use it in the build. gcc is convenient because it supports the -include command-line option that instructs the C preprocessor to include a specific file before processing any other include files. This allows you to include fcgi_stdio.h without modifying Perl source files. (The reason for specifying gcc version 2.7 is that I have experienced bad behavior with an earlier version and the -include flag -- the C preprocessor died with SIGABRT.)

   • $fcgi is used as shorthand for the full path of the FastCGI developers kit.

   If those are valid assumptions, follow these steps:

   1. Pull the Perl source kit from http://www.metronet.com/perlinfo/src/latest.tar.gz

      There are good sources of information on Perl at:

      • http://www.perl.com/
      • http://www.metronet.com/perlinfo/

   2. Unpack the tar file in the parent directory of the FastCGI kit directory, so that the perl directory is a sibling of fcgi-devel-kit. $perl is used as shorthand for the full path of the directory in which Perl is installed.

   3. Copy the version specific and the common files from fcgi-devel-kit/perl-5 into the Perl-5 kit.

      > cd $perl
      > mv perl.c perl.c.orig
      > mv proto.h proto.h.orig
      > mv Configure Configure.orig
      > cp -r ../fcgi-devel-kit/perl-5/perl5.002/* .
      > cp -r ../fcgi-devel-kit/perl-5/common/* .
      

      The files you are copying contain the Perl-5 FCGI extension, some files modified from the distribution, and a script to simplify the configuration process.

   4. Set environment variables. The Perl-5 FastCGI configuration process requires that the environment variable FCGIDIR be set to the top level directory of the FastCGI development kit.

      > setenv FCGIDIR $fcgi
      

      If you do not want to use gcc to build Perl you can set the environment variable CC to the desired compiler. For example:

      > setenv CC gcc2.7
      

      By default Perl's installation prefix is /usr/local, so binaries get installed in /usr/local/bin, library files get installed into /usr/local/lib/perl, etc. If you want to specify a different installation prefix set the environment variable PERL_PREFIX.

      > setenv PERL_PREFIX /usr/local/perl5-fcgi
      

   5. Run fcgi-configure.

      > ./fcgi-configure
      

      fcgi-configure is a wrapper around Perl's Configure script. It sets some variables according the the value of some environment variables, and runs Perl's Configure script in such a way that it does not prompt the user for any input. 90% of the time this should work without a problem. If for some reason this does not work for you, you'll have to follow the steps in the next section.

   6. Run make.

      > make
      

   7. Install the newly built Perl-5.

      > make install
      

   3.2 Semi-advanced recipe

   If you do not have experience configuring and building Perl, you should find someone who does. Perl can be pretty intimidating to configure since it asks you a large number of irrelevant-seeming questions that you won't know how to answer.
   1. Go into the top level directory of the Perl distribution and run Configure.

      > cd $perl
      > ./Configure
      

   2. There are some questions that you are going to have to answer differently when building FastCGI into Perl. These are described below:

      Use which C compiler?

      You should specify gcc.

      Any additional cc flags?

      You should use the following cc flags along with any defaults that Perl Configure supplied:

      • -I$fcgi/include
      • -include $fcgi/include/fcgi_stdio.h

      This assumes you are using GCC.

      Any additional ld flags (NOT including libraries):

      You should specify the following ld flags:

      • -L$fcgi/libfcgi

      Additional Libraries:

      add -lfcgi to the list of additional libraries. It should be added before -lc.

      What extensions do you wish to load dynamically?

      If you can support dynamic extensions, Configure will ask which of the supplied extensions should be loaded dynamically. Since we copied the FCGI extension into the Perl source directory it should be one of the ones in the default list. If you want FCGI to be dynamically loaded you should specify it here, otherwise leave it out.

      What extensions do you wish to load statically?

      If you do not support Dynamic extensions this is the only question about extensions you would get asked. You should specify FCGI here if you did not get asked about dynamic extensions (or did not specify FCGI as a dynamic extension).

   3. Copy in the new proto.h.

      The file proto.h has some macros that conflict with the FastCGI macros. The version of proto.h supplied in the FastCGI kit includes these changes:

      • At the beginning of the file it adds the following lines:

        #ifdef _FCGI_STDIO
        #undef printf
        #endif
        

      • At the bottom it adds:

        #ifdef _FCGI_STDIO
        #define printf FCGI_printf
        #endif
        

   4. Copy in the new perl.c.

      Perl-5.002 has a bug in perl.c that has a great chance of getting exercised with FastCGI. A fix has been sumbitted to the Perl developers and hopefully it'll make it into perl-5.003. It was a one line fix, here is a diff for the curious:

      *** perl.c      1996/03/15 17:10:10     1.1
      --- perl.c      1996/03/15 17:11:23
      ***************
      *** 405,410 ****
      --- 405,411 ----
            if (e_fp) {
      	if (Fflush(e_fp) || ferror(e_fp) || fclose(e_fp))
      	    croak("Can't write to temp file for -e: %s", Strerror(errno));
      +	e_fp = Nullfp;
      	argc++,argv--;
      	scriptname = e_tmpname;
            }
      

      Pretty straightforward.

   5. Build and install Perl.

      > make
      [...]
      > make install
      

      3.3 Advanced recipe

      If you already have a Perl-5 package that has been configured, and you do not really want to re-run Configure, you can take the following steps.

      THIS IS NOT RECOMMENDED

      Edit config.sh with your favorite editor and modify the following lines:

      cc

      Change to use gcc if you are not using it already.

      ccflags AND cppflags

      Add the following flags:

      • -I$fcgi/include
      • -include $fcgi/include/fcgi_stdio.h

      This assumes you are using GCC. See the above section on assumptions

      extensions AND known_extensions

      Add FCGI to the list of extensions

      ldflags

      Add -L $fcgi/libfcgi to the list.

      libs

      Add -lfcgi to the list of libraries, should be added before -lc.

      static_ext or dynamic_ext

      Add FCGI to the list of statically or dynamically loaded extensions.

      d_stdio_cnt_lval, d_stdio_ptr_lval, d_stdiobase, d_stdstdio

      Change all of these to undef.

      One you have edited config.sh you should do a "make Makefile depend all". If you're paranoid like me you may want to do a "make clean" first.

      4. Writing FastCGI applications in Perl

      The Perl program examples/tiny-perl-fcgi performs the same function as the C program examples/tiny-fcgi that's used as an example in the FastCGI Developer's Kit document. Here's what the Perl version looks like:

      #!./perl
      use FCGI;
      $count = 0;
      while(FCGI::accept() >= 0) {
          print("Content-type: text/html\r\n\r\n",
                "<title>FastCGI Hello! (Perl)</title>\n",
                "<h1>FastCGI Hello! (Perl)</h1>\n",
                "Request number ", $++count,
                " running on host <i>$ENV('SERVER_NAME')</i>");
      }
      

      If you've built Perl according to the recipe and you have a Web server set up to run FastCGI applications, load the FastCGI Developer's Kit Index Page in that server and run this Perl application now.

      The script invokes Perl indirectly via the symbolic link examples/perl. It does this because HP-UX has a limit of 32 characters for the first line of a command-interpreter file such as examples/tiny-perl-fcgi. If you run on HP-UX you won't want to sprinkle symbolic links to perl everywhere, so you should choose a PERL_PREFIX shorter than /usr/local/perl5-fcgi.

      You need to be aware of the following bug. If the initial environment to a FastCGI Perl application is empty (contains no name-value pairs) then when the first call to FCGI::accept returns, the environment will still be empty, i.e. %ENV will contain no associations. All the variables associated with the first request are lost. There are two known workarounds:

      • In your Perl application, enumerate %ENV using each before entering the FCGI::accept loop. The program examples/tiny-perl-fcgi contains code for this.

      • In configuring your application be sure to set at least one initial environment variable. You do this with the AppClass -initial-env directive to the Web server, or by running cgi-fcgi in a non-empty environment.

      The Perl subroutine FCGI::accept treats the initial environment differently than the C function FCGI_Accept. The first call to the C function FCGI_Accept replaces the initial environment with the environment of the first request. The first call to the Perl subroutine FCGI::accept adds the variable bindings of the first request to the bindings present in the initial environment. So when the first call to FCGI::accept returns, bindings from the initial environment are still there (unless, due to naming conflicts, some of them have been overwritten by the first request). The next call to FCGI::accept removes the bindings made on the previous call before adding a new set for the request just accepted, again preserving the initial environment.

      The Perl FCGI module also includes subroutines FCGI::finish, FCGI::set_exit_status, and FCGI::start_filter_data that correspond to C functions in fcgi_stdio.h; see the manpages for full information.

      Converting a Perl CGI application to FastCGI is not fundamentally different from converting a C CGI application. You separate the portion of the application that performs one-time initialization from the portion that performs per-request processing. You put the per-request processing into a loop controlled by FCGI::accept.