0
Not a bug

Comics not imported / umlaut mark bug

Emmanuel 4 years ago updated by Sergio Antonio Delgado Quero 3 years ago 37
I found out that Ubooquity will not import comics containing an umlaut mark in the file name. Comics with some Dutch file names like: "Druïden.cbz" or "Magiërs.cbz" are not imported to Ubooquity.
Under review
It looks like a local problem which happens on some types of NAS.
On what are you running Ubooquity ?

I am running a Synology NAS.

Running DSM 5.2
From what I could gather, this is caused by the lack of internationalization support in DSM.

Crashplan (another Java based application) provides a detailed tutorial to solve the problem.

If you decide to apply this fix, please let me know the outcome of the operation. I'll add this link in the FAQ if it solves the non US-ASCII characters problem.
And here is a second tutorial describing the operation in even more details, just in case.

According to the author, the files with special characters have to be created after the internationalization support is installed, unfortunately. This would mean you'd have to copy your files to another machine, then copy them again on the NAS so that it properly takes these characters into account.
The tutorial looks easy. But if I understand correctly, I have to do this every DSM (Synology firmware) update. Sooner or later I will forget. Don't know if I have to copy the comics to another machine every DSM update. But it sounds like to much hassle for just 11 comics containing an umlaut mark.

Best would be if Synology supports non US-ASCII characters. I will make a request to Synology for that.

I will remove the umlaut mark for those 11 comics. Looks like the most hassle free solution till Synology supports non US-ASCII characters.

Thanx for looking in to this.
There is still a modification I haven't tried yet, which (I hope) would allow files with umlauts to be displayed (although the umlaut would probably appear as another character or a question mark).
I'll try to include it in the next release.
I guess it would still be better than the current behavior.
I agree. A question mark in the file name is much better then no file at all. Would your solution work on all non US-ASCII characters? Like: é or ô
I guess so (haven't tested yet).
If you need someone to test on a Synology, don't be shy.

Thanks. :)

For now I'm trying to install DSM on a virtual machine. I'll see how it goes.
+1
Ok, installation done. I have been able to reproduce the problem.
And then fix it by adding the following Java parameter:

-Dfile.encoding=UTF-8

Could you try launching Ubooquity with it and let me know if it solves the problem for you ?
Your command line would be something like that:

java -Dfile.encoding=UTF-8 -jar Ubooquity.jar -webadmin
(you have to be in the same directory as the "Ubooquity.jar" file)
Done. Running a scan of my comics now. This is better then expected. I see comics with an umlaut in the name. No strange character or question mark. Will report on other non US-ASCII characters when scan is done.

Great job. Thanx a lot.
Thanks for your feedback. It will allow me to update the FAQ and the installation instructions for Synology devices so that other users don"t have the same problem.
All non US-ASCII characters I can think of are recognized. I have added your Java parameter to the ubooquity.jar file and made the change permanent.

I think you can best edit the Matthew Sanders tutorial. Last part of the tutorial is about editing the ubooquity.jar file anyway
Parameter stoped working after restart Ubooquity. Do you know how to make your Java parameter permanent?
+1
It has to be in the command line you use to launch Ubooquity.

So if you followed Matthew's tutorial, you have to modify the /etc/init/ubooquity.conf file.
If you want, just post here the script you use to start Ubooquity and I'll indicate what changes to make.
I meant ubooquity.conf instead of ubooquity.jar....typo.

Here I only made changes to the user dir:

exec /var/packages/JavaManager/target/Java/bin/java -Duser.dir='/volume1/prive/Ubooquity' -jar -Xmx1024m /var/packages/Ubooquity/Ubooquity.jar -port 2202 -webadmin



Sorry, I'm quite late. Here is the line modified with the encoding option (and the new "workdir" parameter):

exec /var/packages/JavaManager/target/Java/bin/java -Dfile.encoding=UTF-8 -jar -Xmx1024m /var/packages/Ubooquity/Ubooquity.jar -port 2202 -webadmin -workdir "/volume1/prive/Ubooquity"

thanx a lot. I am testing it now.
I am sorry to say, but it is not working. I don't see non US-ASCII characters in my comic DB.

I stopped Ubooquity. Edited ubooquity.conf to your startup script. Deleted the work/userdir, just to be sure. Started Ubooquity to run a new scan. No non US-ASCII characters to be found.

Did I do something wrong?
Just to be sure to understand: the "UTF-8" parameter worked when you launched Ubooquity manually but not anymore when used in the ".conf" script ?

If so, I'll try it again on my side.
(I will be probably unable to use a computer for a week or two, so please be patient, but don't hesitate to ping me at the end of the month if you don"t get feedback)
Yes you understand correctly. When I go to the ubooquity.jar dir and use the command "java -Dfile.encoding=UTF-8 -jar Ubooquity.jar -webadmin" it works perfectly. Using your startup script edited in the ubooquity.conf file did not work.


Take your time. Glad you are willing to help.


I remember Ubooquity did import non US-ASCII file names on my Synology in the past. I posted a problem with the sorting of comic's starting with the letter  on this forum. But was using an older version of DSM, Ubooquity and Java.
+2

The solution is to manually set the LANG environment variable before executing Java on the init script:


LANG=en_US.UTF-8
export LANG


Here is my full init script for reference:


description "start/stop Ubooquity"
author "lapause"
start on started httpd-user
stop on runlevel [06]
setuid http
script
JAVA_DIR=/volume1/@optware/opt/ubooquity/env/ejre1.7.0_75/bin
WORK_DIR=/volume1/@optware/opt/ubooquity/var
PORT=2202
# $MEM=-Xmx512m
LANG=en_US.UTF-8
export LANG
exec $JAVA_DIR/java -Dfile.encoding=UTF-8 $MEM -jar $WORK_DIR/Ubooquity-1.8.2.jar -port $PORT -workdir $WORK_DIR -webadmin
end script

All my folders and filenames with accentuated characters are now fully recognized when Ubooquity is started from init script.

It didn't work for me in DSM 6.0.


This is my /etc/init/ubooquity.conf:


start on started httpd-user
stop on runlevel [06]
export LANG=en_US.utf8 exec /var/packages/Java8/target/j2sdk-image/bin/java -Dsun.java2d.cmm=sun.java2d.cmm.kcms.KcmsServiceProvider -Dfile.encoding=utf8 -jar -Xmx512m /var/packages/Ubooquity/Ubooquity.jar -port 2202 -webadmin -headless -workdir "/volume1/tebeos/.Ubooquity"


I've tried with "utf8", "UTF8", "UTF-8" and "utf-8", both in LANG and in the JVM parameter. No luck.


This is my locale -a:


C
cs_CZ.utf8
da_DK.utf8
de_DE.utf8
en_US.utf8
es_ES.utf8
fr_FR.utf8
hu_HU.utf8
it_IT.utf8
ja_JP.utf8
ko_KR.utf8
nb_NO.utf8
nl_NL.utf8
pl_PL.utf8
POSIX
pt_BR.utf8
pt_PT.utf8
ru_RU.utf8
sv_SE.utf8
tr_TR.utf8
zh_CN.utf8
zh_TW.utf8


Any ideas? Thank you!


Hello,


Tried to follow the thread and apply the fix on QNAP, but my knowledge is too poor to get it fixed. Any tutorial available for QNAP owners ?


Thanks a lot

No one able to give a hint ?

I guess you should try to ask your question on a QNAP forum, you're more likely to find QNAP "experts" there.

Sorry.

This was still on my to do list. Finally made some time to test this out. I used the script from Christian W from this topic: http://ubooquity.userecho.com/topic/576717-tutorial-install-ubooquity-on-a-synology-nas/


Changed the Java and Work DIR.


Works like a charm. I am happy.


Thanx all for helping me out.

So another one with non US characters in a QNAP nas...


I try to modify the UbooQuity.sh file:

#!/bin/sh

CONF=/etc/config/qpkg.conf
QPKG_NAME="UbooQuity"
QPKG_ROOT=`/sbin/getcfg $QPKG_NAME Install_Path -f ${CONF}`
QPKG_NAME1="QJDK7"
QPKG_ROOT1=`/sbin/getcfg $QPKG_NAME1 Install_Path -f ${CONF}`
LANG=en_US.UTF-8
export LANG
export JAVA_HOME=$QPKG_ROOT1
export JAVA=$JAVA_HOME/bin/java
export PKILL=$QPKG_ROOT/pkill
case "$1" in
start)
ENABLED=$(/sbin/getcfg $QPKG_NAME Enable -u -d FALSE -f $CONF)
if [ "$ENABLED" != "TRUE" ]; then
echo "$QPKG_NAME is disabled."
exit 1
fi
cd $QPKG_ROOT

$JAVA —Dfile.encoding=UTF-8 -jar Ubooquity.jar -webadmin -headless &


;;
stop)
: ADD STOP ACTIONS HERE
$PKILL -f "$JAVA -jar Ubooquity.jar -webadmin -headless"
;;
restart)
$0 stop
$0 start
;;
*)
echo "Usage: $0 {start|stop|restart}"
exit 1
esac
exit 0


But don't works...


Keep trying ;)

+2

Ok, i found a solution!!

So for QNAP users, you have to edit the UbooQuity.sh file and add this two lines:

LANG=en_US.UTF8
export LANG

And modify the command line adding:

-Dfile.encoding=UTF8

The trick was that - for some reason - the UTF-8 is not accepted, and you have to use UTF8 instead...

My file remains like this:


#!/bin/sh
CONF=/etc/config/qpkg.conf
QPKG_NAME="UbooQuity"
QPKG_ROOT=`/sbin/getcfg $QPKG_NAME Install_Path -f ${CONF}`
QPKG_NAME1="QJDK7"
QPKG_ROOT1=`/sbin/getcfg $QPKG_NAME1 Install_Path -f ${CONF}`
export JAVA_HOME=$QPKG_ROOT1
export JAVA=$JAVA_HOME/bin/java
export PKILL=$QPKG_ROOT/pkill
LANG=en_US.UTF8
export LANG
case "$1" in
  start)
    ENABLED=$(/sbin/getcfg $QPKG_NAME Enable -u -d FALSE -f $CONF)
    if [ "$ENABLED" != "TRUE" ]; then
        echo "$QPKG_NAME is disabled."
        exit 1
    fi
    cd $QPKG_ROOT
    $JAVA -Dfile.encoding=UTF8 -jar Ubooquity.jar -webadmin -headless &
    ;;
  stop)
    : ADD STOP ACTIONS HERE
    $PKILL -f "$JAVA -jar Ubooquity.jar -webadmin -headless"
    ;;
  restart)
    $0 stop
    $0 start
    ;;
  *)
    echo "Usage: $0 {start|stop|restart}"
    exit 1
esac
exit 0


Thanks for sharing !

It works fine ! Many thanks Jesus !

It didn't work for me in DSM 6.0.

This is my /etc/init/ubooquity.conf:

start on started httpd-user
stop on runlevel [06]
export LANG=en_US.utf8 exec /var/packages/Java8/target/j2sdk-image/bin/java -Dsun.java2d.cmm=sun.java2d.cmm.kcms.KcmsServiceProvider -Dfile.encoding=utf8 -jar -Xmx512m /var/packages/Ubooquity/Ubooquity.jar -port 2202 -webadmin -headless -workdir "/volume1/tebeos/.Ubooquity"

I've tried with "utf8", "UTF8", "UTF-8" and "utf-8", both in LANG and in the JVM parameter. No luck.

This is my locale -a:

C
cs_CZ.utf8
da_DK.utf8
de_DE.utf8
en_US.utf8
es_ES.utf8
fr_FR.utf8
hu_HU.utf8
it_IT.utf8
ja_JP.utf8
ko_KR.utf8
nb_NO.utf8
nl_NL.utf8
pl_PL.utf8
POSIX
pt_BR.utf8
pt_PT.utf8
ru_RU.utf8
sv_SE.utf8
tr_TR.utf8
zh_CN.utf8
zh_TW.utf8

Any ideas? Thank you!

Finally! It worked!!!


You made my day, good sir. :) Thank you!