• View
• Edit
• History
• Attach
• Print
• Backlinks

PageTopStore

Description

By default, PmWiki stores wiki pages on disk using PageStore objects, which have each page and its complete history in one file. To make this work, the wiki markup of a page is slightly mangled (eg. line feeds are replaced by the string %0a). Reading and editing these files outside PmWiki is not particularly easy. PageTopStore extends PageStore by keeping a second copy of each current page in a separate directory (by default, wikitop.d). The copies kept in this directory have a slightly different format that should be easier to read and edit in an external application.

Moving to PageTopStore and back to PageStore should be completely painless, as both read and use the exact same file format; PageTopStore just has an additional file format that it also uses. Conversions between the formats happen automagically. Mostly this is a matter of pages appearing in the wikitop.d directory as they are read, but there's extra magic to allow these pages to be edited outside PmWiki and for those changes to be properly processed and timestamped by PmWiki.

To install this recipe:

1. back up your wiki just in case things go wrong
2. download pagetopstore.phpΔ to your cookbook directory
3. add the following to your configuration file, adjusted for your configuration:
   

   include_once("$FarmD/cookbook/pagetopstore.php");
   $WikiDir = new PageTopStore( 'wiki.d/{$FullName}', 'wikitop.d/{$FullName}' );

Configuration

The PageTopStore object is initialised with six parameters:

• the "normal" PageStore path format (default: wiki.d/{$FullName})
• the "top" path format (default: wikitop.d/{$FullName}) — this needs to be different from the previous or very bad things might happen
• whether the wiki is writable (default: FALSE, but $WikiDir is hard-coded to be writable)
• any default attributes pages should have (default: NULL)
• the author name used for modifications (default: '[PageTopStore]')
• an array of string replacements to the saved page text (default: array( '&lt;'=>'&amp;lt;', '<'=>'&lt;' ))

To use PageTopStore with the default $WikiDir, use the following:

Additionally, setting $EnablePageTopStoreAutofill = FALSE; will limit the automagic behaviour of the recipe such that pages are written to disk only explicitly.

Usage

For the most part, using PageTopStore should be completely transparent to the user. To help with migrating a site to PageTopStore, an additional action filltop is provided. This will create "top" files to match all pages in a PageStore. To use, add ?action=filltop&ps=YourPageStoreVariableNameHere to the URL of any page on your site, with the part in italics replaced by the name of a global variable pointing to your PageTopStore, such as 'ps=WikiDir'.

File format

As stated above, PageTopStore keeps two separate files for each wiki page. The complete history of the page uses the default PageStore format (actually, it uses the exact same functions as PageStore to read and write these files). The "top" copy is slightly different, consisting of a header and a body, separated by a blank line.

The header is read similarly to how files are read from PageStore, and consists of the non-text information about the page; the exact same fields that are stored in $PCache. The end of the header is indicated by a blank line (ie. the string "\n\n"), similarly to how HTTP requests are parsed. The rest of the file consists of the page text (or more accurately, markup), stored almost exactly as it's seen within PmWiki.

By default, PageTopStore replaces < characters in the saved page text with the string < to prevent scripting attacks in cases where the wikitop.d directory is web-readable and the wiki itself is publicly editable. This should be prevented in any case by your site configuration and an automatically generated .htaccess file. To disable this you may call PageTopStore with the sixth parameter set to FALSE.

Synchronisation

Given that the same information (the current contents of the page) are stored in two separate locations, the possibility exists that these will differ. This is handled as follows. Here, $pagefile is the file with the complete history in PageStore format and $topfile is the new PageTopStore file.

if $pagefile doesn't exist :
    if $topfile exists :
        $pagefile is written from information in $topfile.
    else :
        the page doesn't exist.
else [ $pagefile exists ] :
    if $topfile doesn't exist :
        $topfile is written from information in $pagefile.
    else [ both $pagefile and $topfile exist ] :
        if the full page history is being read (eg. for action=edit or action=diff)
        and the filesystem modification time of $topfile is later than that of $pagefile
        and there's a difference between the page contents :
            $pagefile is updated with the changes made to $topfile.

The only default core actions that will update files in wiki.d are edit, diff, postattr, upgrade, approveurls and approvesites.

Notes

This is a work in progress, partly to be treated as a mockup of a possible replacement for PageStore. It needs more testing and comments.

Having an easily readable format for pages opens up a whole new set of possibilities, some of which may be included in this recipe or others. For example, searching and pagelist generation could be done using external tools. This could be significantly faster, especially if it's assumed that the wikitop.d directory only holds wiki pages and that it's complete.

Listing pages may currently exclude pages for which only a "top" file exists.

I welcome suggestions for a better name than wikitop.d, as well as the current practice of referring to a "top" file.

Release Notes

• 2009-11-27Δ — added user-modifiable author name as per gb's request
• 2009-02-16Δ — removed PHP5 requirement, added < => &lt; replacement
• 2009-02-11Δ — fixes for Oliver Betz's comments: changes to any field provoke update, author of change is marked as '[PageTopStore]'
• 2009-01-22Δ — first public release

See Also

• PITS.01063 line break keeping page store
• PmWikiUsers:2009-January/053338.html line break keeping page store (repost)
• PmWikiUsers:2009-February/053945.html when is a mangled "top" version read back to wiki.d? Bug in vardoc.php

Contributors

• Eemeli Aro

Comments

A few problems

• Even if the wiki is viewed by someone who is not an administrator, it's possible to add ?action=filltop&ps=WikiDir to the current URL. Then if you have made modifications outside a web browser (using a text editor), and haven't edited the file after that, the wiki.d folder won't be synchronized. Therefore, a malicious user could call action=filltop and erase those modifications. A solution could be to write into the wikitop.d folder only is a document in wiki.d is newer than the one in wikitop.d. And / Or to allow this command only from an administrator.
• When I made modifications "outside" a browser, I'd like a way to automatically synchronise the wiki.d and wikitop.d folders, without the need to manually edit individual pages within a browser. Farvardin 2011-12-23

Update all pages from wikitop.d to wiki.d

SteP 2010-02-10: Thank you for this recipe! It's very useful to me, and it's working well, so far. Would it be possible to add an action to update all pages in wiki.d from pages in wikitop.d? Reason: I use PageTopStore on my development server only. When I'm ready to move changes to my production server, I need to go through a tedious sequence of manual steps:

• for each page in wikitop.d (development server), is the page newer than its version in wiki.d?
• if so edit the wiki.d page, so it pulls in the changes from PageTopStore

Only then I'm ready to upload pages from my development server to my production server.
The new action could replace the sequence of manual syncing steps.

• Hi ! I'm testing this script and here it is my first comment : for a next version, I would like to have the author in a config variable (I had to change the cookbook code to set my own wikiname, because I didn't want the default one [PageTopStore]). Thank you for sharing this program. gb November 24, 2009, at 09:05 AM
  Fixed now with new fifth parameter —Eemeli Aro November 27, 2009, at 08:52 AM

Use in conjunction with git (Fast Version Control System)

See my comment at the User Notes page below. How do i contribute my changes ? LukasDiduch May 14, 2011 06:31 PM