WEBP image based comics crank up the memory requirements of Ubooquity

I decided to convert a bunch of my CBZ from containing JPG files to WEBP files, with the idea that it could save space without sacrificing quality. Doing so really cranked up the memory requirements of Ubooquity. Before using WEBP, I was running Ubooquity with 2gig of memory allocated. I had to turn it up to 3gig to keep from getting out of memory errors in the log.

How did I discover this? Well, after converting a lot of JPGs to WEBPs, I added these new folders to Ubooquity and launched a new scan of my folders. When it was done, I went in to look at the results. I found that many of my comics were showing the "no cover found" default cover that looks like an icon of a book, and that a few comics had a cover that was only a line where the cover image should be - as if the process that created the cover thumbnail just quit after 2 or 3 lines of the original image. The majority of the comic covers were fine though. When I went to read a comic with the built-in Ubooquity reader, if that comic had the default book icon, the first image wouldn't show. I would get a grey almost gradient background. Sometimes it would be bright green - like a green screen used in movie making. But sometimes, if I clicked through the page images, I would eventually get to an image that displayed properly. For each page I would get a spinning hourglass at the top, while Ubooquity decoded the WEBP image. For bad pages, it might spin forever. Other times it would spin a bit and then display the grey gradient or green screen. Some images displayed properly, though. So it was quite confusing. I tried to make sense of which CBZ files worked and which didn't. I tried to make sense of which WEBP files worked and which didn't. I couldn't find any rhyme or reason.

So I thought, well, I bet I know how to fix the cover issue. I'll copy the JPG covers from the original files into the CBZs with WEBP images. Tedious, but it worked. A Ubooquity rescan and things were looking good as far as the appearance of the covers went. But there were still problems once I opened a file in the reader.

So I dug into the Ubooquity logs to see if I could find errors. Sure enough, there were out of memory errors, like these:

20201127 11:20:23 [qtp122114483-765] ERROR com.ubooquity.d.b - Request not handled (should never happen, this is a bug !): /comicreader/335598
20201127 11:20:23 [qtp122114483-765] WARN  org.eclipse.jetty.server.HttpChannel - /comicreader/335598
java.lang.OutOfMemoryError: GC overhead limit exceeded
20201127 11:20:34 [qtp122114483-788] ERROR com.ubooquity.d.b - Request not handled (should never happen, this is a bug !): /comicreader/335598
20201127 11:20:34 [qtp122114483-788] WARN  org.eclipse.jetty.server.HttpChannel - /comicreader/335598
java.lang.OutOfMemoryError: GC overhead limit exceeded
20201127 11:20:41 [qtp122114483-763] ERROR com.ubooquity.d.b - Request not handled (should never happen, this is a bug !): /comicreader/335598
20201127 11:20:41 [qtp122114483-763] WARN  org.eclipse.jetty.server.HttpChannel - /comicreader/335598
java.lang.OutOfMemoryError: Java heap space
20201127 11:21:31 [qtp122114483-766] ERROR com.ubooquity.c.a - Could not serve stream with URI: /comicreader/335598
org.eclipse.jetty.io.EofException: null

When I checked the memory by using the "View System Info" button on the Ubooquity admin console's General tab, It said I had memory available, but it also seems to indicate that it was using the max memory:

Max memory:	1820 MB
Free memory:	373 MB
Total memory:	1820 MB

Above is before I increased the memory. Below is after increasing to 3gig:

Max memory:	2731 MB
Free memory:	1875 MB
Total memory:	2400 MB

Those memory readings were taken while I had a WEBP comic opened in the Ubooquity reader and had flipped through a few pages.

Part of the memory issue I'm seeing could also be that I increased my "Items per page" for comics to the max 300. So I'm sure that's using up more memory than my old setting of 24. I'm thinking of increasing it further... I might need to jack up the memory allocated from 3gig to 4gig. That seems extreme, but I'd rather not have the errors.

Lastly, if you look at the errors posted above, I'm getting this error regardless of how much memory I allocate:

Could not serve stream with URI: /comicreader/335598

And that concerns me. I don't know what to look at for that.

Anyway, I just wanted to share this so anyone else looking to play with the WEBP image format would know what they're in for.

Happy Thanksgiving,
Tom