[Scummvm-devel] ENGINE AUTHORS: case-insensitivy and directory handling
max at quendi.de
Sun Aug 3 20:28:06 CEST 2008
Am 01.08.2008 um 15:39 schrieb peres:
> Hi team
Hi Peres, hi all,
before I reply to your post, here's a plea:
Everybody, if you have *any* objections to the plan below, please
voice them. If you keep silent, that's fine, too, but please don't
complain afterwards if we make changes you don't like. You've been
> The recent discussion on the devel mailing list has brought the idea
> unifying the handling of game files and directories in a single API
> sits atop of FilesystemNode and deals with case-sensitivity issues
> for the
> engines. This API would thus replace Common::File in the medium-long
> also providing a way to access files in subdirectories.
Well, it would replace Common::File, which is our current "single" (or
not so single) API which tries to deal with case-sensitivity issues
and other stuff; except that it doesn't properly sit atop FSNode, and
has various other shortcomings, but we already discussed that at
> The concept is to model directories instead of a game file, and give
> engine an instance of the directory where the game is stored. This
> be asked to open a file, for which it would return a readable
> stream, or a
> directory, for which it would return another directory object.
This makes sense, aye.
On a sidenote: The ScummFile class in ScummVM might benefit from such
a change, too: It is a Common::File subclass which allows opening
"subfiles"; i.e. it is a wrapper around an container/archive files, in
which various other files are collected together, and this class
allows transparent access to these files via the same API as
Redux: With a GameDirectory/GameDirectories approach as you propose,
this container access code in SCUMM could probably be remodeled in a
more natural fashion. All we would have to do is to allow subclasses
of GameDirectory to be used, too.
Of course, it would also make it easier to add support for reading
files from ZIP files in a transparent fashion... For theme data etc.,
right now we don't want that for game data.
> Going a bit further, this approach also allows more powerful and
> file searches than the current default directory system -
> implemented as
> static members of Common::File - does. The key is an object that
> manages a
> set of the directories introduced above, together with an array of
> settings that specify how searches should be performed (per
> for example how deep a certain directory can be searched, if hidden
> should be considered, and so on...
Also, these objects should cache the directory contents, making
lookups faster and relieving backends from the burden of doing such
caching. Drawback: changes to the FS may not be noticed. My take: if a
user tinkers with the list of present files while ScummVM is running,
they shall bear the consequences ;).
But of course we could easily flush the file/dir caches at certain
points, too (like when returning to the launcher).
> The attachment contains a quick implementation of both classes I
> in this mail: I called them GameDirectory and GameDirectories. It
> be considered a proof of concept more than an usable piece of code,
> has shortcomings so far (like what to do with FSList returned by
> matchPattern). Anyway, I would be glad if engine authors would drop
> considerations about it. I am also posting on the patch tracker so the
> code is easily reachable in the future.
All in all I like the idea. Some of the refinements that come to mind:
* As you already pointed out, of course we wouldn't want to base this
on Common::File. So, I think we should do as Marcus suggested and add
FSNode::openFileForReading/Writing() methods. It seems to me that we
should do that no matter what else we change. So, I'll commit a dummy
implementation for that ASAP (i.e. one which internally uses
Common::File, too, for now).
* Caching the contents of the given dir is mandatory. Just keep a
HashMap of the files. Easy to do (case insensitive) lookups against
that, too. I think I would lazily create it, the first time somebody
tries to access a file in the dir.
* GameDirectory::getSubDirectory() doesn't support error handling in
its current form. I.e. if you try to do it on a non-existing dir.
Right now it'll just crash, I think. But a typical use case is that we
might want to also search for files in the "resource" or "RESOURCE"
subdir, if it exists. That is not really possible right now.
One quick (and dirty?) solution: Return an empty GameDirectory if the
subdir does not exist.
* In getSubDirectory, I think I wouldn't want to error out on a name
clash. Just print a warning. Better to just run as good as possible
(by only using the first of the matching dirs) than to do a full
crash, isn't it? At least in this case...
* Resource managment: One nice thing about class File is that it
automatically takes care of closing the file and freeing any used
memory when it goes out of scope. With GameDirectory::openFile(), we
loose that nice bit. Well, I guess we can use Common::SharedPtr to
resolve that. I.e. advice engine authors to do
SharedPtr<SeekableReadStream> file = dir.openFile("test.dat");
Of course, they would still have to check the returned value to be non-
zero to detect errors, but that's not worse than current code, I guess.
* Finally, I guess a (pseudo) example of how to *use* GameDirectory
would help people to understand and comment on your proposal much more
than any (lengthy) text would :)
> A final word on the way this can be applied to the code base, provided
> there is a consensus around it. Moving engines from Common::File to
> *anything* different is indeed a massive task (there 134 engine files
> referencing Common::File as of today!), but the process can be
> carried on
> gradually, by having this API in parallel and changing code in steps.
Aye, that's an important thing. Though if we really decide to do that
switch, I am sure it could all be done within a single week, once the
new system is in place (should we go with your approach, that is :).
More information about the Scummvm-devel