Your comments
I don't have 2 TB of comics, only around 50-100 GB. :)
So you did not see any heap errors (or any other kind of memory related error) when Ubooquity becomes unresponsive ?
And when it does, is the java process still alive or did Ubooquity completely stopped working ?
Monitoring memory of Java application without specific tools is not easy as Java will use as much memory as possible inside the limits you gave it (the principle being that there is no point in spending cycles freeing memory if you are not close to the limit).
If you define a 1 GB limit, even the smallest of Java programs might consume a few GB while it could easily live with a 200 MB limit.
So you won't be able to guess much by just looking at memory numbers in your task manager unfortunately.
(and if you had no heap errors, the problem is probably not related to memory at all)
The log file grows up to 1MB, then it is zipped and a new log file is created.
5 zip files are kept, when a 6th one is created, the oldest one is replaced.
This behavior can be overridden by your own configuration.
This tutorial explains how to use an external configuration file.
The settings you'll want to modify are "maxIndex" (max number of kept zip files) in the "rollingPolicy" section and "maxFileSize" (max size of a log files before it is zipped) in the "triggeringPolicy" section.
The full documentation of the logger configuration is available here, but it's not very user friendly. Ask me if you need more info.
As for the logging of read comics, I'll add it.
Agreed, 4 GB was already an insanely high amount of RAM for what Ubooquity does.
Like I said, I have to do some profiling, but it will take time.
One last thing to check on your side: does the first occurrence of the memory error happen on the same file each time ?
I'd like to rule out a problem caused by a specific file (already happened in the past with broken PDF files).
Very, very nice !
I was interested in using Let's Encrypt with Ubooquity but was kind of put off by the relative complexity of the procedure.
This will help a lot, thanks !
If you don't mind, I might include this tutorial in the documentation sometime.
I only had a quick glance at your logs, but, like Elouan, I think your problems are (at least partially) coming from the memory shortage.
Once a memory error has happened, all bets are off. The application can crash right after the error or much later, or even continue running like nothing happened.
This kind of error is critical, you can't expect Ubooquity to run properly as long as they happen.
To prevent them, you can increase the allocated memory using a command line option:
On my side I'll have to do some profiling to look for potential memory leak (or file handles leak), but I don't know when I'll find the time to do it.
Just for my information, how many comics were listed in the database when it crashed (I don't need exact numbers, just an idea) ? And what was the size of the database file ?
The main design goals of Ubooquity are simplicity and ease of use.
That's why Ubooquity is packaged in a single file and does not require any external dependencies (apart from the Java runtime environment) to run.
By the way, although Ubooquity acts as a web servers and uses an internal database, everything exists in a unique process.
There is no daemon, no external database instance, no external web server.
My guess is that pluging Ubooquity to your own database would not make you gain much anyway, a few megabytes of RAM perhaps. Not worth the trouble.
As for the Apache web server, as you guessed, it wouldn't bring anything at all.
It happens to everybody. :)
Customer support service by UserEcho
Interesting.
I guess you are using the GUI (not the web admin page).
So the web server part would be broken whereas the rest of the application is still running...
I'll double check that all critical threads are covered by the exception catching mechanism. But it will be a few weeks/months before I do it, I want to finish and release the online epub reader first.
As for the memory management mechanism, the JVM memory management is consistent, it's just a matter of default strategy. Java's default strategy is to free memory only when you come close to the allowed limit (this is not entirely true for all GC strategies, but still...). This makes sense as freeing memory more frequently would consume resources (and induce latency or add micro freezes) without providing much benefit (as long as you consider memory as a non critical resource).
In any case, a small application like Ubooquity should not require any specific memory configuration, nor should it require more than a few hundred megabytes of memory. I have to investigate and create some test case with massive numbers of files.