[Scummvm-devel] [RFC] ScummVM website re-write

[Max Horn]

Hi again,

since nobody else replied, I'll do that now. Further comments from  
everybody are welcome; if there aren't any, I'll just push this  
through with [vEX] alone.

First off, I welcome a rewrite of the ScummVM website code, primarily  
because the existing code base is rather difficult to extend and has  
accumulated some cruft. Also, it would be nice to switch from a table  
based website layout to a proper CSS layout one day; with the current  
code base, this is very difficult, so separating presentation and data  
sounds very appealing to me.

However, it *does* work well, and so a switch has to be considered  
carefully, and tested beforehand.

Also, I really prefer to keep everything "text-only", i.e. no database  
usage. This way, the complete website is stored in subversion, which  
makes it trivial to backup/restore/replicate it at any time. Luckily,  
[vEX] work seems to be consistent with this.

Another thing to ponder is whether one could use some more "off the  
shelf" components. Like an existing CMS, or at least e.g. a gallery  
software, etc.. Pro: Saves us coding, gives us powerful features for  
free: Like being able to add & edit news items via a webbrowser.
Con: Most of these require DBs, which I would prefer to avoid, or at  
least the advantage would have to far outweigh the advantage of not  
using a DB (see above).

However, we could still use some stuff. The proposal already involves  
smarty. We could use some other 3rd party tools. E.g. for the  
screenshots, we could use SPGM <http://spgm.sourceforge.net/>.

Another thing I mentioned: I'd like to retain the existing URLs, at  
least for the most important pages (screenshots, downloads, faq,  
compatibility, ...). For backwards compatibility, but I also prefer  
human readable (and meorizable!) URLs in any case. That shouldn't be  
an issue, though.

One more idea: It might be nice to be able to provide a localized  
website one day, i.e. with german/french/... pages. Nothing to worry  
about right now, but it would not hurt to at least keep that in  
mind... for the very distant future... ;)

Now some specific comments to the RFC:


[...]
> +-=--        --=-+
> | I. Pros / Cons |
> +-=--        --=-+
>
> Pros:
> - New code base, rewritten from scratch keeping logic, data and  
> presentation
>  separated.
> - Should adhere to the ScummVM Code Formatting Conventions (let me  
> know if I
>  missed something).
> - Uses XML for storing data, being a standard it can be reused  
> easier than
>  storing data in PHP variables.
> - A bonus from separating out presentation is that the template  
> engine (Smarty)
>  supports caching, which should speed things up a bit. [1]
> - The new codebase will work under PHP5 (there are some minor things  
> in the old
>  codebase that will not work under PHP5), which is more futureproof  
> as PHP4
>  will reach it's end of support on 8th august 2008.
> - PHPDoc commented code (similiar to JavaDoc), should make it easier  
> to follow
>  the code.

To me, most of the above "Pros" are "only" nice boni, but not enough  
to warrant a switch. But enabling CSS usage, and making it easier to  
extend the codebase, might buy me ;).

Caching is certainly nice from a theoretical POV, but in reality, we  
never had any speed issues with the site. As long as it doesn't  
complicate the code, fine by me, but no need to jump through hoops to  
get it.

[...]

> Separating out the presentation and putting it in templates was  
> something that
> really needed to be done. The current codebase is one heck of a mix  
> between
> PHP and HTML that's not particulary easy to follow.

See above, fully agreed

> The template engine of
> choice is Smarty [2] as I've got previous experience using it, and  
> it supports
> caching. Being able to cache the rendered result should speed things  
> up a bit.
> It should be more I/O than CPU processing when caching is enabled.

If we get caching "for free", great.


> For those of you who has read about design patterns, you should  
> hopefully see
> that this is me trying to follow the MVC (Model-View-Controller) [1]  
> pattern.

MVC is one of the oldest and proven methods, I am certainly happy to  
use it.

> Since I'm using XML, and the simple XML-parser doesn't speak HTML I  
> have had use
> something else. I've opted for using BBCode since I think it's easy  
> enough to
> use, and most people should be familiar with it already.

An alternative approach would be to just add a select subset of HTML  
to the DTD, no? Or even import that HTML DTD... BBCode might be easy  
to use, but HTML is so, too....

[...]

>
> The css and templates directories contains all stylesheets  
> respectively
> templates. The data directory, and it's subdirectories contains all  
> the data
> used (XML-files, screenshots and media files). The downloads  
> directory should
> contain the downloads not available as "normal" SourceForge files,  
> ie: daily
> SVN builds and subproject SVN builds. The include directory contains  
> the
> 'config.inc' file, the Factory classes and the Utils class. It also  
> contains
> a small PHP-file called 'emulate_php5.php' containing simulated  
> functions that
> are available in PHP5 but not PHP4.

Is this there to support running on PHP4 systems? Does the site fully  
work with PHP4 then? (Not so important, as SF.net is our primary host,  
and I do not forsee us moving to a host which does not support PHP5,  
ever ;).

[...]

> What's done:
>
> Currently all the items under 'Misc. Menu' are done. Under the  
> 'Documentation'
> menu group I've got Compatibility done, as well as FAQ/Credits but  
> since those
> just read the already generated HTML they don't really count. The  
> documentation
> is also pretty much done, except for the 'SCUMM Data File MD5  
> Checksums' which
> also reads some pre-generated content. I'd rather write a parser for  
> each of
> these that reads from the source directly.

Makes perfect sense.

> That way data/presentation separation
> can be reached for those pages as well. For the 'Main Menu' I've got  
> the home
> page and all the news reading done. Though it still reads the old  
> format as I've
> not converted them over.
>
> What's missing:
>
> Well, plenty of stuff really. I haven't touched the screenshots page  
> since the
> data is at the moment stored in two different places. The  
> screenshots category
> data is stored in PHP, while the information about the images and  
> their captions
> are stored in a CSV-file. I'm not sure what to do with this, but  
> personally I
> don't think it's nice to have half the data stored in one place and  
> the other
> half in another. I could make it all XML if that is desired, or if  
> there are
> some really good reasons to keep the CSV-file I could just put the  
> category
> information into XML.

This has mostly historical reasons, I'd say. This code and the system  
certainly could be improved a lot. Also note: You do not have to be  
pixel-level compatible here. And one could use a 3rd party gallery  
tool for some of this, e.g. SPGM <http://spgm.sourceforge.net/>, or  
something else.

[...]

> The news is still handled by reading the old format. I would however  
> like to
> change those to real XML as well. That would however mean to change  
> all the
> HTML-tags to BBCode (or something else) at the same time. That's  
> something I
> don't want to do until everyone have voiced their opinions.

I would be fine with changing the format of the news. In fact, the  
whole news system could be a lot better. I wouldn't mind switching to  
a completely different "news" system, either, maybe even one "off the  
shelf". As long as it can use text files for input... ;).


> BBCode parsing isn't enabled at the moment, I'm thinking of creating  
> some
> abstract class which all objects should inherit from. In that class  
> I'd put a
> function that given an array it would check the keys and set the  
> class variables
> where there is a matching key. That function would also make sure to  
> parse the
> data first.

There are lots of BBCode parsing classes out there; if we really want  
to use BBcode, one should at least try to use of them. Or just use  
some existing "news" system, as suggested above.

>
>
>
> +-=--             --=-+
> | V. How to handle... |
> +-=--             --=-+
>
> All the pre-rendered content is something I'd like to get rid off.  
> I've looked
> at the credits.pl script and wouldn't mind hack it to make XML for  
> the website
> instead of HTML. The --html switch could become --xml-website (and  
> the --xml one
> could be changed to --xml-docbook) to reflect the change.

No problem.

>
>
> I found a faq.xml in docs/trunk/docbook, as well as a faq-inc.xsl,  
> is that what
> is used to generated the faq.inc file? That one could be difficult  
> to use, at
> least with the XML-parser as it would pick up all the docbook tags  
> as XML-tags.
> I could write some code that makes the XML-parser ignore a given  
> list of tags
> and that should hopefully be enough

Actually, we use faq-inc.xsl (an XSL style sheet) to translate DocBook  
XML to XHTML. In the same vein, it should be possible to write an XSL  
file which translates it to any other XML variant you prefer. So I  
don't see the problem here.

I am not sure what you mean with "it would pick up all the docbook  
tags as XML-tags" -- we are using DocBook XML, so those  
"tags" (actually, the correct term is "elements", to be nitpicky ;-)  
*are* XML-elements... So again, I don't see the problem?


> .
>
> +-=--           --=-+
> | VI. Some thoughts |
> +-=--           --=-+
>
> This is a small collection of some of my thoughts on the current  
> code and how
> some small things could change.
>
> The could should work fine on PHP4, I have PHP5 installed and that's  
> what I've
> tested against. However I've made sure to only use what's available  
> in PHP4. But
> there might of course be something that I've missed that prevents it  
> from
> working. Report all such issues so I can take care of those.
>
> At the moment all the different pages have their own *Page class,  
> which is
> unnecessary. The reason for this is that at the beginning I was too  
> eager to
> start coding that I didn't go through all the code. Unless anyone  
> have any
> strong opinions to keep them separated I'm going to merge them into  
> one Page.php
> file as different functions. The Utils::switchPage() function will  
> just call on
> those functions instead.

Please merge them.

>
>
> On the news page, the date is formatted using PHP's date() function,  
> however
> Smarty only supports the strftime() options. That's fine for  
> everything, except
> where the english ordinal suffix (-st, -nd, -rd, -th) is used.  
> Currently I'm
> using a third-party Smarty plugin, but I rather generate the correct  
> output in
> the Factory and add it to the object. I think it's better than to  
> depend on
> the plugin, which might stop working with future Smarty versions. Or  
> the file
> might get deleted when updating Smarty.


Actually, I couldn't care less about the precise date format used in  
the news. In fact, personally, I prefer ISO format anyway (YYYY-MM- 
DD). Using a plugin just to get this to be compatible with the current  
code is total overkill ;). Also, using a 3rd party news system might  
resolve that, too.


> I might have taken the OO-part a bit to the edge by making objects  
> out of pretty
> much everything. Just look at the SubprojectsFactory, the  
> Subprojects-class
> only contains an info string and an array with Project-objects. It  
> could easily
> have been represented with an array containing the same information  
> instead.

Then maybe consider doing that? I am pro-OO and pro-MVC; but   
overdesign and code bloat can be quite painful, so .... ;)


Bye,
Max