index.py

Show log
Commit

Hash : 6842ee81
Author :
Date : 2012-08-17T09:58:38

More cleanups to the documentation part of libxml2

doc/Makefile.am:

* Build what's in doc/ before doc/devhelp/, as the dependency graph flows
   that way

* Add "--path $(srcdir)" so that xsltproc can find DTDs in srcdir

* Replaced $(top_srcdir)/doc with an equivalent $(srcdir)

* Qualified libxml2-api.xml with $(srcdir) as it's always generated there

* Rewrote the dependencies for libxml2-api.xml so that xmlversion.h
   doesn't throw everything off

doc/devhelp/Makefile.am:

* Use Automake constructs to install the HTML files instead of an
   install-data-local rule

* Reorganized the file a bit (hello whitespace!)

* EXTRA_DIST doesn't need to list so many files now that dist_devhelp_DATA
   is being used

* Only print "Rebuilding devhelp files" if rebuilding is actually
   occurring

doc/examples/index.py:

* Make the "this file is auto-generated" banner more prominent

* Autotools updates: Use AM_CPPFLAGS/AM_CFLAGS instead of INCLUDES

* Got rid of DEPS as it's not needed (Automake already sees the dependency
   on libxml2.la by way of LDADD(S))

* Replaced LDADDS with LDADD, which is applied to all programs listed
   in the file. Since all the test programs have the same link
   dependencies, this way is more concise yet equivalent.

* Remove the *.tmp files via "make clean" instead of having the test
   programs do it themselves (more on this later)

* Invoke index.py in srcdir, as it pretty much needs to run there

* Restructured the index.html rule so that only the xmllint invocation is
   allowed to fail

* Use $(MKDIR_P) instead of $(mkinstalldirs), $(VAR) instead of @VAR@

* Remove symlinks for test?.xml in an out-of-source build

* Sort lists for neatness

* Better formatting for EXTRA_DIST and noinst_PROGRAMS variables

* Simplified the Automake bits printed for each program: *_LDFLAGS doesn't
   need to be specified as it's empty anyway, *_DEPENDENCIES is redundant,
   *_LDADD isn't needed due to the global LDADD

* Added a bit that symlinks in test?.xml from srcdir in out-of-source
   builds. This allows the reader4 test to read these files in the current
   directory, which ensures that the output always looks the same (i.e.
   does not contain references to srcdir)

* Don't hide the test program invocation (or else it's hard to tell which
   test failed), and don't use superfluous parentheses

* NOTE: If you check in these changes, be sure to run this script and also
   check in the updated files that it generates!

doc/examples/*.c:

* Updated the test: lines so that

   + "&&" is used to separate commands instead of ";" so that errors are
     not masked

   + reference files are qualified with $(srcdir)/

   + no "rm" takes place -- these are a problem because (1) if a test
     fails, it's useful to have the output file ready for inspection; (2)
     the "rm" invocation masks a potential non-zero exit status from diff

     (This is why I added the CLEANFILES line above)

doc/examples/io1.res:

* Updated this ref file so that the test passes. (This is correct, right?)

doc/examples/reader4.res:

* Changed this back to its original form, as the symlinking of test?.xml
   means this file no longer has to contain path prefixes on the filenames

doc/examples/testWriter.c:

* Changed the output filenames to *.tmp instead of *.res, partly for
   consistency, partly to not have to add special cases to CLEANFILES

doc/examples/xpath1.c:

* Removed the "./" prefix on the test invocation, which is redundant as
   index.py already adds one

Download

doc/examples/index.py

#!/usr/bin/python -u
#
# Indexes the examples and build an XML description
#
import string
import glob
import sys
try:
    import libxml2
except:
    sys.exit(1)
sys.path.insert(0, "..")
from apibuild import CParser, escape

examples = []
extras = ['examples.xsl', 'index.py']
tests = []
sections = {}
symbols = {}
api_dict = None
api_doc = None

def load_api():
    global api_dict
    global api_doc

    if api_dict != None:
        return
    api_dict = {}
    try:
        print "loading ../libxml2-api.xml"
        api_doc = libxml2.parseFile("../libxml2-api.xml")
    except:
        print "failed to parse ../libxml2-api.xml"
	sys.exit(1)

def find_symbol(name):
    global api_dict
    global api_doc

    if api_doc == None:
        load_api()

    if name == None:
        return
    if api_dict.has_key(name):
        return api_dict[name]
    ctxt = api_doc.xpathNewContext()
    res = ctxt.xpathEval("/api/symbols/*[@name = '%s']" % (name))
    if type(res) == type([]) and len(res) >= 1:
        if len(res) > 1:
	    print "Found %d references to %s in the API" % (len(res), name)
	node = res[0]
	typ = node.name
	file = node.xpathEval("string(@file)")
	info = node.xpathEval("string(info)")
    else:
        print "Reference %s not found in the API" % (name)
	return None
    ret = (typ, file, info)
    api_dict[name] = ret
    return ret

def parse_top_comment(filename, comment):
    res = {}
    lines = string.split(comment, "\n")
    item = None
    for line in lines:
        while line != "" and (line[0] == ' ' or line[0] == '\t'):
	    line = line[1:]
        while line != "" and line[0] == '*':
	    line = line[1:]
        while line != "" and (line[0] == ' ' or line[0] == '\t'):
	    line = line[1:]
	try:
	    (it, line) = string.split(line, ":", 1)
	    item = it
	    while line != "" and (line[0] == ' ' or line[0] == '\t'):
		line = line[1:]
	    if res.has_key(item):
	        res[item] = res[item] + " " + line
	    else:
		res[item] = line
	except:
	    if item != None:
	        if res.has_key(item):
		    res[item] = res[item] + " " + line
		else:
		    res[item] = line
    return res

def parse(filename, output):
    global symbols
    global sections

    parser = CParser(filename)
    parser.collect_references()
    idx = parser.parse()
    info = parse_top_comment(filename, parser.top_comment)
    output.write("  <example filename='%s'>\n" % filename)
    try:
        synopsis = info['synopsis']
	output.write("    <synopsis>%s</synopsis>\n" % escape(synopsis));
    except:
        print "Example %s lacks a synopsis description" % (filename)
    try:
        purpose = info['purpose']
	output.write("    <purpose>%s</purpose>\n" % escape(purpose));
    except:
        print "Example %s lacks a purpose description" % (filename)
    try:
        usage = info['usage']
	output.write("    <usage>%s</usage>\n" % escape(usage));
    except:
        print "Example %s lacks an usage description" % (filename)
    try:
        test = info['test']
	output.write("    <test>%s</test>\n" % escape(test));
	progname=filename[0:-2]
	command=string.replace(test, progname, './' + progname, 1)
	tests.append(command)
    except:
        pass
    try:
        author = info['author']
	output.write("    <author>%s</author>\n" % escape(author));
    except:
        print "Example %s lacks an author description" % (filename)
    try:
        copy = info['copy']
	output.write("    <copy>%s</copy>\n" % escape(copy));
    except:
        print "Example %s lacks a copyright description" % (filename)
    try:
        section = info['section']
	output.write("    <section>%s</section>\n" % escape(section));
	if sections.has_key(section):
	    sections[section].append(filename)
	else:
	    sections[section] = [filename]
    except:
        print "Example %s lacks a section description" % (filename)
    for topic in info.keys():
        if topic != "purpose" and topic != "usage" and \
	   topic != "author" and topic != "copy" and \
	   topic != "section" and topic != "synopsis" and topic != "test":
	    str = info[topic]
	    output.write("    <extra topic='%s'>%s</extra>\n" % (
	                 escape(topic), escape(str)))
    output.write("    <includes>\n")
    for include in idx.includes.keys():
        if include.find("libxml") != -1:
	    output.write("      <include>%s</include>\n" % (escape(include)))
    output.write("    </includes>\n")
    output.write("    <uses>\n")
    for ref in idx.references.keys():
        id = idx.references[ref]
	name = id.get_name()
	line = id.get_lineno()
	if symbols.has_key(name):
	    sinfo = symbols[name]
	    refs = sinfo[0]
	    # gather at most 5 references per symbols
	    if refs > 5:
	        continue
	    sinfo.append(filename)
	    sinfo[0] = refs + 1
	else:
	    symbols[name] = [1, filename]
	info = find_symbol(name)
	if info != None:
	    type = info[0]
	    file = info[1]
	    output.write("      <%s line='%d' file='%s' name='%s'/>\n" % (type,
	                 line, file, name))
	else:
	    type = id.get_type()
	    output.write("      <%s line='%d' name='%s'/>\n" % (type,
	                 line, name))
	    
    output.write("    </uses>\n")
    output.write("  </example>\n")
    
    return idx

def dump_symbols(output):
    global symbols

    output.write("  <symbols>\n")
    keys = symbols.keys()
    keys.sort()
    for symbol in keys:
        output.write("    <symbol name='%s'>\n" % (symbol))
	info = symbols[symbol]
	i = 1
	while i < len(info):
	    output.write("      <ref filename='%s'/>\n" % (info[i]))
	    i = i + 1
        output.write("    </symbol>\n")
    output.write("  </symbols>\n")

def dump_sections(output):
    global sections

    output.write("  <sections>\n")
    keys = sections.keys()
    keys.sort()
    for section in keys:
        output.write("    <section name='%s'>\n" % (section))
	info = sections[section]
	i = 0
	while i < len(info):
	    output.write("      <example filename='%s'/>\n" % (info[i]))
	    i = i + 1
        output.write("    </section>\n")
    output.write("  </sections>\n")

def dump_Makefile():
    for file in glob.glob('*.xml'):
        extras.append(file)
    for file in glob.glob('*.res'):
        extras.append(file)
    Makefile="""##
## This file is auto-generated by index.py
## DO NOT EDIT !!!
##

AM_CPPFLAGS = -I$(top_builddir)/include -I$(top_srcdir)/include -I$(srcdir)/include
AM_CFLAGS = $(THREAD_CFLAGS) $(Z_CFLAGS)
LDADD = $(RDL_LIBS) $(STATIC_BINARIES) $(top_builddir)/libxml2.la $(THREAD_LIBS) $(Z_LIBS) $(ICONV_LIBS) -lm $(WIN32_EXTRA_LIBADD)

CLEANFILES = *.tmp

rebuild: examples.xml index.html

examples.xml: index.py $(noinst_PROGRAMS:=.c)
	cd $(srcdir) && $(PYTHON) index.py
	$(MAKE) Makefile

index.html: examples.xml examples.xsl
	cd $(srcdir) && xsltproc examples.xsl examples.xml && echo "Rebuilt web page"
	-cd $(srcdir) && xmllint --valid --noout index.html

install-data-local: 
	$(MKDIR_P) $(DESTDIR)$(HTML_DIR)
	-$(INSTALL) -m 0644 $(srcdir)/*.html $(srcdir)/*.c $(srcdir)/*.xml $(srcdir)/*.xsl $(srcdir)/*.res $(DESTDIR)$(HTML_DIR)

clean-local:
	test -f Makefile.am || rm -f test?.xml

"""
    examples.sort()
    extras.sort()
    tests.sort()
    EXTRA_DIST=""
    for extra in extras:
        EXTRA_DIST = EXTRA_DIST + " \\\n\t" + extra
    Makefile = Makefile + "EXTRA_DIST =%s\n\n" % (EXTRA_DIST)
    noinst_PROGRAMS=""
    for example in examples:
        noinst_PROGRAMS = noinst_PROGRAMS + " \\\n\t" + example
    Makefile = Makefile + "noinst_PROGRAMS =%s\n\n" % (noinst_PROGRAMS)
    for example in examples:
        Makefile = Makefile + "%s_SOURCES = %s.c\n\n" % (example, example)
    Makefile = Makefile + "valgrind: \n\t$(MAKE) CHECKER='valgrind' tests\n\n"
    Makefile = Makefile + "tests: $(noinst_PROGRAMS)\n"
    Makefile = Makefile + "\ttest -f Makefile.am || test -f test1.xml || $(LN_S) $(srcdir)/test?.xml .\n"
    Makefile = Makefile + "\t@(echo '## examples regression tests')\n"
    Makefile = Makefile + "\t@(echo > .memdump)\n"
    for test in tests:
        Makefile = Makefile + "\t$(CHECKER) %s\n" % (test)
        Makefile = Makefile + '\t@grep "MORY ALLO" .memdump | grep -v "MEMORY ALLOCATED : 0" ; exit 0\n'
    try:
	old = open("Makefile.am", "r").read()
	if old != Makefile:
	    n = open("Makefile.am", "w").write(Makefile)
	    print "Updated Makefile.am"
    except:
        print "Failed to read or save Makefile.am"
    #
    # Autogenerate the .cvsignore too ...
    #
    ignore = """.memdump
Makefile.in
Makefile
"""
    for example in examples:
        ignore = ignore + "%s\n" % (example)
    try:
	old = open(".cvsignore", "r").read()
	if old != ignore:
	    n = open(".cvsignore", "w").write(ignore)
	    print "Updated .cvsignore"
    except:
        print "Failed to read or save .cvsignore"
    
if __name__ == "__main__":
    load_api()
    output = open("examples.xml", "w")
    output.write("<examples>\n")

    for file in glob.glob('*.c'):
	parse(file, output)
	examples.append(file[:-2])

    dump_symbols(output)
    dump_sections(output)
    output.write("</examples>\n")
    output.close()
    dump_Makefile()

Download

kc3-lang/libxml2/doc/examples/index.py

Commit

doc/examples/index.py