[Scummvm-devel] File handling revamp

Max Horn max at quendi.de
Tue Jul 29 16:03:10 CEST 2008


Am 29.07.2008 um 02:30 schrieb Johannes Schickel:

>> * The overly convoluted, complicated, under-documented and abused
>> File::addDefaultDirectory() system (The blame is on me to a big  
>> extent, of
>> course :). Semantics are unclear here, too. Right now, we blindly  
>> add every
>> directory in our reach and it's friends, and then just hope the  
>> files we
>> want are included somewhere there. And that there are no name  
>> clashes. No
>> way of saying "game data files are in these dirs, GUI theme stuff  
>> please
>> only from that dir". One specific problem: The AdvancedDetector's
>> detectGame() function needs to support taking either a FSList  
>> param, or
>> alternatively uses File::open() to locate files. Besides  
>> complicating the
>> code, this causes a disparity between detecting and running games,  
>> which can
>> lead to subtle bugs.
>
> Well there's one problem all engine developers have to address when
> using FSNode for opening files in directories: case (in-)sensivity.
> That blows the client code unnecessarily IMHO.

You are completely right.

<RANT>But I wish people would point this out immediately when they run  
into it, instead of silently suffering or using other approaches ...</ 
RANT>

So, this is an issue. Luckily, one that shouldn't be hard to fix. E.g.  
by augmenting the FSNode API to support case insensitive filenames, or  
by providing simple wrapper which hides these details from engines.  
Tell me where and how you need the case insensitivity, and we'll  
implement it ASAP :-).



> Of course
> addDefaultDirectory is case sensitive too AFAIK,

Actually it is mixed: The param it takes is case sensitive, but the  
filename cache it then creates is case insensitive. How weird is  
that.. :/

> but it's less work to
> do: addDefaultDirectory("foo"); addDefaultDirectory("FOO"); than using
> the FSNode code to detect what case the file has.

Well, you simply don't want to have code detecting that, I guess. All  
you want is to say "Gimme the dir 'foo'", and then you don't care  
about the case. In 99% of cases, there is only one dir foo or FOO, and  
you should just get that (either via an overhauled/added FSNode API,  
or suitable wrapper code). Personally, I'd tend to say: "Screw the  
folks who keep both a "resource" and a "RESOURCE" folder in their game  
data copy). Seriously, I don't think anybody should have that. Of  
course we could also add ways to compensate for that, but I don't  
think it's worth the effort.

> Also for getting
> FSNode objects to files inside that path FSNode one has to handle
> possible cases too, or creating an FSList and compare names via an
> case insensitive comparator, which would be utterly stupid and hacky
> :-).

Depends on what you mean...
Iterating over an FSList using a case insensitive comparator is  
precisely what most of our code walking dir contents currently does.  
It seems to be the most natural thing to do when you want to, you  
know, try to search a list, not caring for case... :-).

However I am talking about game detection code here, which is looking  
for multiple files at once. If you just want to open (during play  
time) a single file "bar.dat" inside some directory "foo", then of  
course you should not be forced to walk an FSList, agreed. You just  
want to do something like
   f.open("foo/bar.dat");
or
   f.open(FSNode("foo/bar.dat"));
or
   f.open(FSNode("foo").getChild("bar.dat"));
or
   f.open("bar.dat", mySearchPaths);
or even just
   f.open("bar.dat");	// Search in the global list of default seach  
paths

The first two simply represent convenience accessors, easy to  
implement by writing a simple parser (relative) for POSIX style  
passes, which would then internally create the corresponding FSNodes,  
in a case insensitive fashion. I.e. they would both amount to  
something like the third example, which in turn of course requires us  
to enhance FSNode accordingly. Should be no big deal, though (note: If  
dir "foo" does not exist, FSNode("foo") represents an invalid node,  
getChild would then also return an invalid node, and File::open would  
finally fail because of this, as expected).

The alternative (or maybe not alternative, but rather complement) is  
to use search dirs, as discussed below. And yeah, we could still have  
some globaly active search dirs. But one of my points is that I think  
it's better to not jumble *all* the files in possibly a dozen dirs  
into one big list, but rather try to be a more bit selective about  
it :).

Depending on the requirements of the client code, each of these code  
have its merits. Personally, I would think that for client code  
writers the following are the most important possibilities:
1) f.open("foo/bar.dat");
2) f.open("bar.dat");
3) f.open("bar.dat", mySearchPaths);
What is unclear to me is whether the first one should only look under  
PATH/foo/bar.dat; should look in all search paths for a dir foo with a  
file bar.dat.



> That 'issue' is known since some time though and the KoM engine and
> Parallaction engine had/have problems with it. Peres told me
> Parallaction doesn't handle case sensitivity at all (yet). It only
> affects BRA currently as far as I got it though, so nothing we have to
> worry about for our next release.

Sadly it was not known to me :/. I recommend writing about such  
problems to this list for discussion. Talking about it on IRC only  
reaches a limited audience.

[...]

> I only answered to the, in my opinion, most important points in this
> mail. I hope nobody thinks I do not care about the others, but I'm
> getting a bit tired so maybe that will follow later.

I greatly appreciate that, no matter how much / little you wrote :-)


Cheers,
Max




More information about the Scummvm-devel mailing list