00447: Distinguish !Page and Category.Page in backlinks

Summary: Distinguish !Page and Category.Page in backlinks

Summary was: Categories Not Indexed

Created: 2005-06-06 06:25
Status: Closed: added for 2.3.0
Category: Feature
From: XES
Priority: 5544443 33
Version: 2.2
OS: pmwiki.org

Description: On PmWiki.org, I noticed that Category.Category is not showing many category entries. For example, "Blog", "Bundles", etc. which are used in the Cookbook area. In fact, it seems to be working backwards and only showing the items that are linking to pages, rather than items listed as in a category.

"Search" is the category used on Cookbook/SearchExtensions, but it shows up under GroupFooter category -- because it has a link to Category.GroupFooter on the page, not because it's in the GroupFooter category.

This means the site's nifty "indexing" feature is broken.

The feature that was originally requested exists and can be enabled via $AutoCreate. --Petko February 26, 2010, at 05:53 PM

Allow distinction between !Page and Category.Page in backlinks

Adding to this PITS entry, rather than create a new one, since I think these are related

2010-01-10 reopened to see if we can achieve some action on the following:

There is a clear distinction between

  • a list of pages which refer to a category group (ie use [[Category/example]] somewhere in the page)
  • a list of pages which are categorised (using the [[!example]] markup in the page)

When referring to "categories" it would be useful to be able to distinguish between [[!Example]] (which is declarative) and [[Category/Example]] (which is referential).

Consequently one could then create a pagelist of all declared (ie defined or categorised) pages, or a pagelist of all pages on which a Category is referenced (ie the Category/Question page), as opposed to all pages on which a category is referenced

Unfortunately (:pagelist link=Category.Example:) picks up both types of category references.

This change request is to provide a means of selecting between these two

I second the need for a more nuanced category listing method. I would also like to see a category listing which includes "sub" categories... skumar? January 31, 2008, at 12:07 PM

Looks like there is a solution using intermap link, as discussed there :

This may be a work around, but it does not address the original issue of [[!Category]] having a different meaning (its a tag) from normal link markup. simon November 12, 2009, at 03:05 AM

I agree with Simon that this is just a work around. It complicates the usage of this way to organize information. For example it requires a different intermap for every instance of PmWiki. At least one for pmwiki.org site CategoryPmWikiOrg: and one for your local site CategoryLocalSite:. This is not easy for, and lowers the value of this feature for, the entry level of users of PmWiki; a group of users that not directly will start and acquire the knowledge to use pagelists. -- SchreyP November 12, 2009, at 12:37 PM

I am experimenting towards adding this feature. I feel that the markups:

  • link=!Name should list only pages containing [[!Name]], same for link=~User
  • link=Category.Name should continue to list all pages linking to the category, both those containing [[!Name]] and those containing [[Category.Name]].

This would make sure that existing wikis using the current link=Category.Name will not break by skipping pages containing [[!Name]]. Otherwise some (most?) upgrades will require huge amount of work. Comments will be welcome.

BTW, the required changes to the core are not trivial and we (especially Pm) may decide to not apply them for the moment. --Petko February 26, 2010, at 09:58 PM

Petko, this is a most welcome development, and the solution is elegant, and fits the 'PmWiki way'™. I hope that we will see this in time, given the amount of work. simon February 27, 2010, at 12:32 PM

Hi Petko, Nice to see you found a solution that up to some level is downwards compatible. I hope the amount of work is on an acceptable level too. I still believe in the value of this simple feature. -- SchreyP February 27, 2010, at 05:11 PM

The changes are now available for testing, see PmWiki:ChangeLog and get them at PmWiki:Subversion. Feedback welcome, especially eventual bug reports, mess with other markups, etc. Note that you may need to remove your existing wiki.d/.pageindex file to allow PmWiki to re-index the wiki with the new functions.

This should also allow very easy creation of link prefixes similar to !Category and ~Profile, like @Tag or *Project:

  $LinkPrefixGroup['@'] = 'Tags';     # [[@Link]] = [[Tags/Link]]
  $LinkPrefixGroup['*'] = 'Projects'; # [[*Link]] = [[Projects/Link]]

And these pages are found the same way, link=@TagName finds only pages with [[@TagName]], link=Tags.Name finds pages with both [[@Name]] and [[Tags/Name]]. --Petko February 27, 2010, at 10:31 PM

I have suspended this request because of technical difficulties for a globally logical solution, per [1]. We'll deal with it when we find a better way to do it. --Petko March 05, 2010, at 03:25 PM

I'll just add that the more I think about it the more I think this markup:

(:pagelist link=\!Mycat:)

is a pretty intuitive markup, particularly to anyone who has any technical background at all. If nothing else it is 100% consistent with *so* many other languages, scripts, syntax elements, etc from other technical areas. In C, Bash, Sh, Csh, PHP, PCRE regex, Perl, etc., etc. (even CSV format!) the backslash character is used to remove special meaning from a character and make it stand simply for itself. Right now the bang (!) has the special meaning of negating the current search element -- if we used this suggested markup then all we would be doing is using the backslash to remove that special meaning in that particular context.

For anyone with any programming background (or even importing a CSV file) this markup is going to be very intuitive. For non-technical authors it may take a quick explanation. The advantage of going with a "proven" syntax like this is that we have good precedent for it being very extensible in the future -- if it works in languages as complex as the list above then we can be pretty confident it is a well-designed syntactical element that we're not going to regret later. Personally I think it's worth it for the small amount of documentation that would need to be added...

But I'm biased, so I'll limit my thoughts to this PITS entry rather than the mailing list.

Petko, are the changes that you had proposed available somewhere? I looked in SVN and didn't see any tags that looked relevant...

--Peter Bowers March 06, 2010, at 06:53 AM

You can get the intermediate revision 2531 via SVN. --Petko March 06, 2010, at 07:46 AM
svn export -r r2531 svn://pmwiki.org/pmwiki/trunk

And what about following suggestion:

(:pagelist link=!/Mycat:) - Putting the "/" after the symbol.

This suggests more intuitive that we are talking about a link and not a negation; better? -- SchreyP March 06, 2010, at 08:32 AM

How about this:

(:pagelist link=+!Mycat:) - using an explicit "+" if starting with !

--Peter Bowers March 06, 2010, at 03:57 PM

To me, the markup link=\!, link=+! looks bad and is too complicated. But did you know that '!' in pagelists is a new feature added recently (01176)? Note that I'm not for or against the things discussed in this entry, but all that Peter Bowers said sounded reasonable to me. --rogutes March 07, 2010, at 09:11 AM

I think the differentiation between marking a page as having a category, and referencing a page in the Category group is important, and some means of distinguishing the two should exist. How about a new pagelist property, category= which shows only pages using the [!xxx] markup, and different to the link=... which shows the both the Category group pages and pages using the category markup. If not, then I prefer the link=\!Name idea.

DaveG March 07, 2010, at 10:43 AM

The differentiation between membership creation and listing can be partially achieved with available PmWiki means - simply use two pages resp. groups:

  • One "linking" page - linking to this page creates category membership. This page should not exist!
  • A "listing" page containing the page list and additional information for the category.

In contrast to the interwiki workaround mentioned above, links to the listing page are still real wiki links, therefore you can find backlinks to this page IOW pages referring to the category but not being members of this category.

How to do it:

1. In config.php, add:

$CategoryGroup = 'CategoryLinks'; # just an example name
$FmtPV['$CategoryGroup'] = '$GLOBALS["CategoryGroup"]';

2. Adapt the page lists to link={$CategoryGroup}.whatever

This solution has a drawback, if the category listing page does not exist: Since only [[!foo]] links are handled by the "temporary workaround for blank category pages" in stdmarkup.php, a link to a non-existing listing page displays the question mark, and opens the page in edit mode.

Neither Peter Bowers' extension below nor the $AutoCreate capability work around this problem.

This problem is not specific to this "virtual group" solution - it also applies to the request above to differentiate between the two link formats in pagelists!

Another issue is that following a [[!foo]] link doesn't lead to the listing page with additional information about the category, and he might try to add information to the link page. Therefore he has to be (re-)directed to the to the listing page. This can be done with a group customization file:

<?php if (!defined('PmWiki')) exit();
Redirect('Category.' . PageVar($pagename, '$Name'));

Not exactly a beautiful solution till now.

OliverBetz 2010-03-09

This below can be a great way to have the functionality (Distinguish !Page and Category.Page in backlinks) without breaking anything else. This should be converted to a Cookbook recipe. --Petko July 08, 2013, at 12:43 AM

Suggestion by Peter Bowers:

Possible solution to the above drawback:

In config.php:

$CategoryGroup = 'CategoryLink';
$FmtPV['$CategoryGroup'] = '$GLOBALS["CategoryGroup"]';
if (preg_match("/^Category[.\/]/", $pagename)) {
  $DefaultPageTextFmt = '';
  SDV($PageNotFoundHeaderFmt, 'HTTP/1.1 200 Ok');

This allows non-existent pages in the group 'Category' to appear without error messages or "Create X" messages. (This is already accomplished for the CategoryLink group with almost identical code in scripts/stdmarkup.php.) All that remains, then, is to populate those pages with a pagelist via the footer.

Then CategoryLink.GroupFooter needs to be created with this source (or something similar):

(:pagelist link={*$FullName} list=normal:)

and Category.GroupFooter needs to be altered to this (note the link=CategoryLink.{*$Name}):

(:Summary:content generated for Category list pages:)
(:pagelist link=CategoryLink.{*$Name} list=normal:)

This whole idea is a prototype currently... The proof-of-concept now exists...

It is time to reopen and complete this issue. The blocker of the use of bang (!) in expressions should be removed by deprecating its use, since the feature only recently added and is unlikely to be used widely if at all.

simon June 14, 2010, at 06:16 PM

Something Hans suggested back in March was a (:pagelist category=Foo:) as contrasted to (:pagelist link=Category.Foo:). The first (a new addition to pagelist) would pick up [[!Foo]] while the second would remain consistent with current usage, finding occurrences of BOTH [[Category.Foo]] AND [[!Foo]].

Or, alternatively, (:pagelist link=Category:Foo ...:) could refer only to pages containing [[!Foo]]. It seems like I've seen a recipe around implementing Category:Foo as a way to join a category (as a member) ... or maybe that's another wiki? Anyway, it's not too bad in terms of intuitive. (As opposed to some of my previous suggestions [above] which look pretty painful in retrospect... :-) )

-Peter Bowers June 19, 2010, at 09:22 AM

Any progress on these technical difficulties? TIA

simon July 20, 2019, at 02:20 AM

In a page data file references and declarations of categories are clearly captured in the data line:


Perhaps initially simply having a config.php option that restricts only declarations [[!Jam]] markup to be added to targets and omits references [[Category/]] markup would be useful, or better yet, starting a new data line for only declarations [[!Jam]] markup


would be better, leaving existing functionality as is, and adding different markup to pagelist, i.e. category= instead of link= (but giving it the functionality of PageListMultiTargets)

simon April 12, 2020, at 11:26 PM