cross-reference/INSTALL

This INSTALL file is customized for the version of MXR used by
mozilla.org. MXR is derived from LXR, the original LXR can be
found at:
   http://lxr.linux.no/


In order to install MXR, you will need:

 - Perl version 5 or later.
 - A webserver with cgi-script capabilities.

and optionally, to enable the freetext search queries:

 - Glimpse


If you don't have Perl installed, get it from
<URL:http://www.perl.com/perl/info/software.html>.

If you need a webserver, take a look at Apache at
<URL:http://www.apache.org/>

If you want Glimpse and the freetext searching facilities, visit
<URL:http://glimpse.cs.arizona.edu/>.


MXR works on GNU/Linux and Solaris with Apache and NES.
Other unix-like operating systems and decently
featured webservers should work as well.


To install MXR itself:

 - Retrieve the mxr source with hg from hg.mozilla.org:
   hg clone http://hg.mozilla.org/webtools/mxr

 - Edit lxr.conf to fit your source code installations and needs.

 - Make sure the files in http can be reached via your webserver.
   Make sure your webserver executes the files search, source, ident
   and diff as cgi-scripts.  With the Apache webserver this can be
   accomplished by making .htaccess contain the following lines:

       <Files ~ (search|source|ident|diff)$>
       SetHandler cgi-script
       </Files>

   You may also need to edit these Apache config files appropriately
        srm.conf
            AddHandler cgi-script .cgi

        access.conf
            Options <whatever> # setting to all will work
            AllowOverride Options

   Its also recommended to set up a robots.txt file on your server to
   discourage robots from traversing your source tree and performing
   an identifier lookup for each identifier in your code. This will
   take up a lot of cpu time and probably several gigabytes of files.

 - Generate the identifier database.  Go to the directory you
   configured as "dbdir" in lxr.conf and do "genxref foo", where foo
   is the subdirectory containing the actual source code.

 - (Optional) Generate the Glimpse database.  Go to the directory you
   configured as "dbdir" in lxr.conf and do "glimpseindex -H . foo",
   where foo is the same as above.  You might want to add other options
   to the commandline (e.g. "-n"), see the Glimpse documentation for details.


If it doesn't work:

 - Make sure all the permissions are right.  Remember that the
   webserver needs to be able to access most of them.

 - Check that all the Perl scripts find their library files, also when
   executed by the webserver.


Multiple Trees

 There are two ways to configure MXR to index multiple trees.
 The first way relies on configuring virtual host/alias support in
 your web server. This turns out to be fairly painful for dynamic
 systems where you're likely to add trees to MXR frequently and
 your web admin does not like making frequent changes to the web
 server's configuration.

Virtual Host based configuration for Multiple Trees

 This requires a web server (such as apache) that supports virtual
 hosts and/or aliases. At mozilla.org, we had set the document root
 of lxr.mozilla.org to the lxr/root directory and for each tree,
 and set up an alias with the tree name pointing to the main mxr
 directory.

 From our httpd.conf:
 <VirtualHost 207.200.73.38:80>
        DocumentRoot /opt/webtools/lxr.mozilla.org/root
        ServerName lxr.mozilla.org
        Alias /classic /opt/webtools/lxr.mozilla.org
        Alias /ef /opt/webtools/lxr.mozilla.org
        Alias /grendel /opt/webtools/lxr.mozilla.org
        Alias /mailnews /opt/webtools/lxr.mozilla.org
        Alias /mozilla /opt/webtools/lxr.mozilla.org
        Alias /nspr /opt/webtools/lxr.mozilla.org
        Alias /seamonkey /opt/webtools/lxr.mozilla.org
 </VirtualHost>

Symlink based coniguration for Multiple Trees

 This requires a file system which supports symlinks (ntfs junctions
 should also work). This is the approach currently used by
 mxr.mozilla.org.

 Create symlinks for each root in the MXR directory:
ln -s . ef
ln -s . nspr
ln -s . js
ln -s . webtools

 The more formal way to do this is now:
./add-root.pl ef /data/mxr/ef/mozilla ef
./add-root.pl nspr /data/mxr/nspr/mozilla nspr
./add-root.pl seamonkey /data/mxr/seamonkey/mozilla mozilla
./add-root.pl mozilla-central /data/mxr/mozilla-central/mozilla-central mozilla

 add-root.pl will both create the symlink and add the appropriate lines
 to lxr.conf. The third optional argument is roughly speaking the tree's
 "pretty" name.

Currently root/index.html is not automatically generated,
it could and should be.

Files you will need to change manually:
 update-src.pl
  - add pull/checkout rules for each tree
 root/index.html
  - advertise your roots with this file which is the user facing root
    of the MXR install

Files you can customize:
 template-*
  - these files are used to piece together the output from
   * find
   * search
   * ident
   * source
   * diff
 Local.pm
  - add support for VCS integration magic
  - add special directory prefix handling
 index.html
  * this is the root for all trees

Scripts you will want to run from a cron job (in order)
 ./update-src.pl -cron treename &&
 ./update-search.pl -cron treename &&
 ./update-xref.pl -cron treename

The easiest way to add a tree to your new (or growing) MXR:
 ./add-root.pl treename /path/to/tree/root treeprefix
e.g.:
 ./add-root.pl bugzilla /data/mxr/bugzilla/webtools/bugzilla mozilla/webtools/bugzilla

the result:
 sourceroot: bugzilla /data/mxr/bugzilla/mozilla/webtools/bugzilla
 sourceprefix: bugzilla mozilla/webtools/bugzilla

Sometimes you may want to change the location of your trees,
without having to rerun the indexer (which can be prohibitively
expensive). The most common case for this is when you're
importing a tree from another file system and the paths don't match.

The easiest way to move a tree on your file system:
 mv /path/to/tree/root /newpath/to/tree/root
 ./update-root.pl treename /newpath/to/tree/root

The more common variant:
 [origin mxr-data] $ rsync tree/root host:/newpath/to/tree/root
 [host mxr] ./add-root.pl treename /path/to/tree/root
 [host mxr] ./update-root.pl treename /newpath/to/tree/root

The argument to add-root isn't a valid path on host, but that's
ok, it's only necessary so that update-root.pl can understand
how to change the path to match the new file system layout.

The lxr.conf file:

 MXR does not care much about your directory structure, all relevant
 paths can be configured from the lxr.conf file. This file is located
 in the same directory as the perl script files.

 MXR recognizes the following options in the configuration file:

 baseurl
  The url for the root directory of your source. Tree name (if any)
  is appended to this.


 htmlhead
  The header of all html files. This is a template that
  contains mainly html code but it can also contain some special
  directives, these are documented below.

 sourcehead
  A special version of htmlhead used for source code listings.

 sourcedirhead
  A special version of htmlhead used for directory listings.

 htmltail
  Template for bottom of pages.

 htmldir
  Template file for the directory listings.

 sourceroot
  The root directory of the source to be indexed. If you're indexing
  several version you could include a variable in the path.
      sourceroot: /usr/local/lxr/source/$v/linux/

  To index more than one tree, include one sourceroot entry for
  each tree. Each entry is a tree name/directory pair.
      sourceroot: classic /export2/lxr-data/classic/mozilla

 virtroot
  This is prepended to the path name when forming the url in links.

 bonsaihome
  If bonsai is set up for your code then set this as the url prefix. If not,
  then remove the bonsai related code in the htmlhead file and ignore this.
  More info on bonsai can be found at http://www.mozilla.org/bonsai.html

 sourcerootname
  The name of the root (more....)

 sourceprefix
  The prefix between the global prefix and the starting point for the root.

 incprefix
  Where to find source specific include files.

 dbdir
  Where to find the database files that lxr needs (fileidx xref and
  the glimpse files). The tree name (if any) is appended to this.

 alias
  Defines another tree with the same properties as an existing alias/tree.

 glimpsebin
  Location of the glimpse binary on your system.

 variable
  This defines a variable that can be used in templates and
  the config file. The syntax is
    variable: <name>, <text>, <values>, <default>
  <name> is the name of the variable, <text> is a textual description,
  <values> are the possible values of the variable.
  <default> is the default value of the variable.

  The <values> field can either be a list starting with a "(" and
  ending with a ")", with elements separated with ",", or it can be
  [ <filename> ]. In this case the values are read from a file with
  one value on each line.

  EXAMPLE:
  # Define typed variable "v", read valueset from file.
  variable: v, Version, [/local/lxr/source/versions], [/local/lxr/source/defversion]

  # Define typed variable "a".  First value is default.
  variable: a, Architecture, (i386, alpha, m68k, mips, ppc, sparc, sparc64)

 map - This makes it possible to rewrite directories using variables.
  The linux sourcecode for instance contains several different
  architectures, the include files for each of these are found in the
  directory /include/asm-<architecture>/. To remap each of these
  according to a variable $a you can specify
    map: /include/asm[^\/]*/ /include/asm-$a/
  Find creative uses for this option :-)