[scummvm-devel] RFC: What should happen when adding \0 to a String?

[Torbjörn Andersson]

Hi,

A while back, there was some work done on unifying the String classes. 
I'm not familiar with the details of that, but apparently as a 
consequence of that some subtle behavior changed. The one In know about 
is adding a \0 character to a String, both whether it should be allowed 
at all and whether it should be counted as part of the String's length.

This caused some problems (I know I fixed one in Myst), and a workaround 
was added so that adding \0 to a String does nothing. Unfortunately, 
there is at least one engine (ADL) that absolutely depends on this 
feature, and is now broken. This affects at least Hi-Res Adventures #5 
and #6.

So the question here (and I really have no opinion myself) is, should it 
be possible to store \0 as part of a String, and if so exactly what 
should happen? The options I can think of are:

- Nothing should happen. (This is the current behavior.)
- It should be allowed, but it should terminate the string. (I.e. 
strings work like in C.)
- It should be allowed, and it should not terminate the string. (I.e. 
strings can contain arbitrary binary data.)

But maybe there is some other even more sensible behavior?

Of course, whatever behavior we decide on we need to make sure 
everything works with it. At sev's suggestion, I've added a warning() to 
base-str.cpp to notify when \0 is added to a String. I have also gone 
through every game I've added to ScummVM to see whether or not it 
triggers the warning just by starting it and walking around for a bit.

The results of my test can be found in 
http://www.update.uu.se/~d91tan/tmp/string.txt

Note that some of the games I started are not even supported, and that 
there are some game engines where I don't own a single game. My list is 
far from exhaustive.

Torbjörn Andersson