From what I have seen, usually a few thousand dollars or euros.
That's not realistic (nor reasonable for a small project like Ubooquity).

Epub reader is on the way (though the way is quite long), and PDF.js integration is next.

It depends on the total memory available on your system as well as the version of the JVM you use.
It's printed somewhere at the beginning of Ubooquity log file if you want to take a look.

Yes, you can try that (for 500MB of memory, change it as you like):

nohup java -Xmx512m -jar Ubooquity.jar -webadmin </dev/null &>/dev/null &

I spotted an error in the script, probably the one that prevented you from using it, I'll fix it tonight. Done.

Unfortunately, apart from adding CPUs and RAM, there is not much you can do. Ubooquity was not designed to support heavy loads (the initial use case is a small home server or a NAS).

The only thing you can try (if you did not already) is to give Ubooquity more RAM to work with.
This is done with the -Xmx Java option.
If you use the script I provide, just uncomment this line (remove the "#") and change the value if needed:

#MEM_OPT=-Xmx512m

I think the problem you had was fixed by the recompression step, not the deletion of the .DS_Store file.

What field would be use for sorting ? Publication date ?

The page split is done using quite convoluted CSS properties combinations.
I am not satisfied with the way it is done, I'll very probably rewrite this part once I have found a more reliable way to split pages using Javascript and CSS (I'm thinking "canvas").