Dear Tom

The memory leak problem is still there on Linux Synology NAS DSM 6.1 with Ubooquity 1.10.1

Consequently the server crashes although the process is not killed but the server remains unusable

Therefore I wonder if the leak is really Windows specific.

Maybe a temporary approach could be within the ubooquity code to force the java [Unix/Windows] FileSystemProvider to close all files that were parsed when loading a library page each time a new library page is loaded ?

Could you please workaround the leak and release a fix in the version 1.10.1 so that we have a fully stabilized version in the meantime you release 2.1 ?

...20170501 21:03:42 [pool-1-thread-6] ERROR com.ubooquity.f.d - Could not list files in folder:
java.nio.file.FileSystemException:

/volume1/distr/Books/Comics/Library/FR ± Soleil/Arleston [S] # Opale
(02-001-2013) [Codex] ¤ (FAN,HER,TEE): Too many open files

at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) ~[na:1.8.0_102]
   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.8.0_102]
   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[na:1.8.0_102]
   at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427) ~[na:1.8.0_102]
... etc

Thank you very much in advance

All the best

Regards

...

For the concern about metadata I have tested version 2.0.2 and I can also confirm it is working as expected.
Thank you so much for having considered keeping this feature.

All the best
Regards

Eric

Hello Tom

I would also support Matthieu's request to let possible using metadata instead of file name at least like what we have with version 1.0. I have a very big comic book library > 300 000 files where changing the folder names and the folder structure will really be a problem. I use scripts that can convert *.cb* files into pdf with automatic metadata generation where the title as metadata automatically includes the issue number so that everything can be properly displayed by Ubooquity v1.x without having to change my file / folder structure.

When you have a large library it is not really practical to put too much information into the file name as it is then quite hard to manage changes that could apply to multiple files. Therefore using metadata to display while storing files with technical names is much more efficient.

Could you please make sure to keep possible with Ubooquity v2.0 the feature of using metadata instead of file name to display title data ?

WIthout this feature I would have to stay with Ubooquity v 1.0 and will not be able to benefit from the new improvements you are planning as well as the fixes you already have to apply to v 2.0 beta.

If that is not possible could you please at least issue a stabilized "final" version of ubooquity v1.0 with all the remaining v1.0 fixes including the "memory leak" issue crashing the server when too many files are opened.

Thank you very much in advance for this excellent software which has no equivalent in its area.

All the best
Regards

Eric
.

Hello Tom

I have made some progress and identified 2 interesting bugs around this matter :

- 1) when the PDF "Description" tag (Dublin Core property) contains one or more NewLine (NL) or Carriage Return (CR) characters the content is not imported anymore by Ubooquity. If I remove the NL/CR characters the content is imported by ubooquity after a rescan

- 2) With the PDF format, he PDF tag "Author" can also be filled by multiple comma separated values. When this happen each single value is stored in the PDF metadata as a distinct "creator" (Dublin Core property). When I make a PDF with such multiple comma separated values in the "Author" tag, the content is not anymore imported by Ubooquity. If I rewrite the "Author" tag without storing the data in multi-valued state the content is visible in ubooquity after a rescan.

Can I send you 2 sample PDF files (one for each bug) so that you can reproduce these issues ?

Thanks Tom for confirming this and for planning it to be fixed. I also reproduced the but myself : when the PDF property "description" is empty, the current version of ubooquity is showing "null" instead of not displaying any text as it should do. When the PDF property "description" is containing text, Ubooquity will normally display this text.
About the need to rescan I fully understand, and will do once the new version of ubooquity will be out.
Thanks again for this nice software ...

Just sent you a 1-page sample PDF "v01 # ALB 001 (2011) (Physic).pdf" to the e-mail address you gave me for you to troubleshoot.

It has been generated by imgpdf v0.1.6 (Python script Johannes 'josch' Schauer) without any PDF properties defined during PDF generation

This PDF was scanned by the latest version of Ubooquity (1.10.1 built on 2016-05-10 at 20:36)

Let me know your findings : Maybe it is normal because no properties were defined and if it this because of that let me know which PDF property I should define when generating the PDF

Regards

I have successfully installed the latest version (Version 1.10.1 built on 2016-05-10 at 20:36) of Ubooquity on a Synology NAS with Java 8 1.8.0_102 using 1024m of Memory (which seems to be enough)

All my comics are PDFs (generated by o script from CBR/CBZ files) with currently no properties/metadata defined in each PDF.

On each book I am seeing the 'null' text displayed as the description for each comic.

Which PDF metadata/property should I fill-in with text --- when generating each PDF --- to remove these 'null' values displayed by Ubooquity ?

Thanks in advance for your help.

Here an updaded version of the "/etc/init/ubooquity.conf" that is now fully working with DSM 6.0 and Java 8 for Ubooquity Version 1.10.1

The script previously shown in the forum does not work anymore with DSM 6.0 because the apache service name (formerly httpd-user) is now replaced by "nginx" thus explaining why the Ubooquity jar was never launched in DSM 6.0 each time the NAS restarts

Please also make sure to use Java 8 and to specify a correct part in the JAVA_DIR argument as well as specifying enough memory for $MEMS

There are 2 a bugs with the automatic font extraction which is triggered each time the jar is lauched
1) the jar attempts to extract them to the directory from which the jar is called regardless of the -workdir parameter value

2) the font extraction attempt is always triggered when the jar directory does not contain the extracted fonts even if they are already extracted in the directory specified by the -workdir parameter value

These bugs can be workarounded by using the below script

I personally prefer to use the jar dir as the work dir but if you want to use another directory just replace "$PCKG_DIR" by "$WORK_DIR" in the instruction starting by exec at the end of the script

If you want to do this, please also make sure to apply this change only after the fonts have been already extracted in the jar dir (i.e. $PCKG_DIR) so that the automatic font extraction will not be triggered anymore.

By doing like this you should not see any font extraction errors in the log each time start the Ubooquity jar is launched.

#/etc/init/ubooquity.conf

# automatically starts ubooquity after apache (DSM 5.0 = httpd-user, DSM 6.0 = nginx) has been started
start on started nginx

# stop ubooquity on
stop on runlevel [06]

# Example
# /var/packages/Java8/target/j2sdk-image/jre/bin/java -Dfile.encoding=UTF-8 -jar -Xmx1024m /var/packages/Ubooquity/Ubooquity.jar -port 2202 -webadmin -headless

script
JAVA_DIR=/var/packages/Java8/target/j2sdk-image/jre/bin
WORK_DIR=/volume1/distr/Books/Ubooquity
PCKG_DIR=/var/packages/Ubooquity
PCKG_BIN=Ubooquity.jar
PORT=2202
MEMS=1024m
ENCO=UTF-8
LANG=en_US.$ENCO
export LANG
CURR_DIR=$(pwd)
cd $PCKG_DIR
exec "$JAVA_DIR/java" -Dfile.encoding=$ENCO -jar -Xmx$MEMS $PCKG_DIR/$PCKG_BIN -port $PORT -webadmin -headless -workdir "$PCKG_DIR"

# exec "$JAVA_DIR/java" -Dfile.encoding=$ENCO -jar -Xmx$MEMS $PCKG_DIR/$PCKG_BIN -port $PORT -webadmin -headless -workdir "$WORK_DIR"

cd $CURR_DIR
end script