View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0002109||2 - Next Dev List (Holding Area)||Bug||public||2017-07-07 13:50||2020-07-02 02:13|
|Platform||Intel i7-5960X||OS||Windows 10 Professional x64||OS Version||1703|
|Summary||0002109: Canada.Country lookup is too large to load in memory|
Data in the Canada.Country module is stored as an XML resource. The resource is about 125 megs in its ASCII representation. It must be converted to Unicdoe to be fed to the XML parser, then shredded into an in-memory data structure.
The Unicode representation would be approximately 250 megs. The in-memory representation is probably 25% of that size, but the Unicode representational remains in memory intil it is completely digested -- the peak memory usage is about 250 * (1 + 0.25) == 312 megs, then. This should fit in a 32-bit process, but is substantial ... and since the XML data is monolithic, it is hard to manage.
A good solution would be to not use XML; then also not use the XML parser. Combined, that means the resource will be substantially smaller and won't need to be converted to Unicode monolithically. The smaller representation can be kept as a static resource where it won't cause a memory allocation, and the data structure built from that representation.
|Steps To Reproduce|
1) Manually build the Canada.Country DLL
2) Make sure it's next to the HRDLogbook.EXE (same directory)
3) Run HRDLogbook.EXE
4) Place a break point on the parser thread function; it's called "LoadCitiesProc"
5) Connect to a cluster.
6) When the break point is hit, you can step through the code to find the code loads the resource, and fails to convert it to Unicode because memory can't be allocated for the conversion (a 250-meg request). The code tries to recover from this error by allocating a CStringW that holds about 125 milllion characters -- another 250-meg memory allocation request. That fails with an unhanded exception from CString and the app crashes.
|Tags||No tags attached.|