is there a size limit on database?

sorry, forgot to say, running on synology 918+ with os version 6.2

Maby not much of help but i have indexed 30000+ on Windows 10 with java 8.

Running fine wih +104000 books.

so how do i get in touch with the developers to investigate the database corruption i am experiencing?

thanks.

adi

Did this problem happen several times ?

i tried this morning to click on clear books database and restart the scan, somewhere in the middle the errors have started again so, yes it does happen several times:

20180731 10:01:39 [Scanner thread] INFO
com.ubooquity.fileformat.pdf.b - Reading metadata of pdf file:
/volume1/Books/2.7K IT Books/Model-Based Development_ Applications.pdf
20180731 10:01:40 [Scanner thread] INFO
com.ubooquity.data.database.a - Inserting/updating book entry in
database : /volume1/Books/2.7K IT Books/Model-Based Development_
Applications.pdf
20180731 10:01:40 [Scanner thread] ERROR com.ubooquity.data.database.a - Error while creating/updating book in database
org.h2.jdbc.JdbcSQLException:
General error: "java.lang.IllegalStateException: Reading from
nio:/volume1/homes/Ubooquity/.Ubooquity2/ubooquity-5.mv.db failed; file
length -1 read length 768 at 4085534 [1.4.193/1]"; SQL statement:
merge
into book(parentid, rootid, isdirectory, title, norm_title, authors,
norm_authors, description, language, norm_language, rating, seriesname,
norm_seriesname, seriesindex, filedate, filepath, filename,
norm_filename, identifier, publisher,
lastupdatedate,filesize,publicationdate,pagecount)
key(filepath)values(?, ?, ? ,? , ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?) [50000-193]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.message.DbException.get(DbException.java:168) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.message.DbException.convert(DbException.java:295) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.Command.executeUpdate(Command.java:266) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]

at
org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:201)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at com.ubooquity.data.database.a.a(SourceFile:987) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.database.a.a(SourceFile:59) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.feeder.a.a(SourceFile:221) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.feeder.a.a(SourceFile:305) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.feeder.a.b(SourceFile:105) [Ubooquity.jar:2.1.1]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_161]
Caused
by: java.lang.IllegalStateException: Reading from
nio:/volume1/homes/Ubooquity/.Ubooquity2/ubooquity-5.mv.db failed; file
length -1 read length 768 at 4085534 [1.4.193/1]
at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:765) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:435) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.FileStore.readFully(FileStore.java:98) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Page.read(Page.java:190) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.MVStore.readPage(MVStore.java:1954) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.MVMap.readPage(MVMap.java:736) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Page.getChildPage(Page.java:217) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Cursor.min(Cursor.java:129) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Cursor.hasNext(Cursor.java:36) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]

at
org.h2.mvstore.db.TransactionStore$TransactionMap$1.fetchNext(TransactionStore.java:1388)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.db.TransactionStore$TransactionMap$1.(TransactionStore.java:1384) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]

at
org.h2.mvstore.db.TransactionStore$TransactionMap.keyIterator(TransactionStore.java:1379)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at
org.h2.mvstore.db.TransactionStore$TransactionMap.keyIterator(TransactionStore.java:1367)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.db.MVSecondaryIndex.find(MVSecondaryIndex.java:309) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.db.MVSecondaryIndex.find(MVSecondaryIndex.java:254) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.index.BaseIndex.find(BaseIndex.java:128) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.index.IndexCursor.find(IndexCursor.java:169) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.table.TableFilter.next(TableFilter.java:467) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.dml.Update.update(Update.java:102) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.dml.Merge.merge(Merge.java:154) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.dml.Merge.update(Merge.java:106) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.CommandContainer.update(CommandContainer.java:98) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.Command.executeUpdate(Command.java:258) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
... 7 common frames omitted
Caused by: java.nio.channels.ClosedChannelException: null
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110) ~[na:1.8.0_161]
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:721) ~[na:1.8.0_161]
at org.h2.store.fs.FileNio.read(FilePathNio.java:74) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:421) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
... 28 common frames omitted
20180731 10:01:40 [Scanner thread] INFO com.ubooquity.fileformat.pdf.b - Reading metadata of pdf file: /volume1/Books/2.7K IT Books/model-based_visual_tracking.pdf
20180731 10:01:40 [Scanner thread] INFO
com.ubooquity.data.database.a - Inserting/updating book entry in
database : /volume1/Books/2.7K IT Books/model-based_visual_tracking.pdf
20180731 10:01:40 [Scanner thread] ERROR com.ubooquity.data.database.a - Error while creating/updating book in database
org.h2.jdbc.JdbcSQLException:
General error: "java.lang.IllegalStateException: Reading from
nio:/volume1/homes/Ubooquity/.Ubooquity2/ubooquity-5.mv.db failed; file
length -1 read length 768 at 4085534 [1.4.193/1]"; SQL statement:
merge
into book(parentid, rootid, isdirectory, title, norm_title, authors,
norm_authors, description, language, norm_language, rating, seriesname,
norm_seriesname, seriesindex, filedate, filepath, filename,
norm_filename, identifier, publisher,
lastupdatedate,filesize,publicationdate,pagecount)
key(filepath)values(?, ?, ? ,? , ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?) [50000-193]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.message.DbException.get(DbException.java:168) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.message.DbException.convert(DbException.java:295) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.Command.executeUpdate(Command.java:266) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]

at
org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:201)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at com.ubooquity.data.database.a.a(SourceFile:987) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.database.a.a(SourceFile:59) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.feeder.a.a(SourceFile:221) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.feeder.a.a(SourceFile:305) [Ubooquity.jar:2.1.1]
at com.ubooquity.data.feeder.a.b(SourceFile:105) [Ubooquity.jar:2.1.1]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_161]
Caused
by: java.lang.IllegalStateException: Reading from
nio:/volume1/homes/Ubooquity/.Ubooquity2/ubooquity-5.mv.db failed; file
length -1 read length 768 at 4085534 [1.4.193/1]
at org.h2.mvstore.DataUtils.newIllegalStateException(DataUtils.java:765) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:435) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.FileStore.readFully(FileStore.java:98) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Page.read(Page.java:190) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.MVStore.readPage(MVStore.java:1954) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.MVMap.readPage(MVMap.java:736) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Page.getChildPage(Page.java:217) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Cursor.min(Cursor.java:129) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.Cursor.hasNext(Cursor.java:36) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]

at
org.h2.mvstore.db.TransactionStore$TransactionMap$1.fetchNext(TransactionStore.java:1388)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.db.TransactionStore$TransactionMap$1.(TransactionStore.java:1384) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]

at
org.h2.mvstore.db.TransactionStore$TransactionMap.keyIterator(TransactionStore.java:1379)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at
org.h2.mvstore.db.TransactionStore$TransactionMap.keyIterator(TransactionStore.java:1367)
~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.db.MVSecondaryIndex.find(MVSecondaryIndex.java:309) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.db.MVSecondaryIndex.find(MVSecondaryIndex.java:254) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.index.BaseIndex.find(BaseIndex.java:128) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.index.IndexCursor.find(IndexCursor.java:169) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.table.TableFilter.next(TableFilter.java:467) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.dml.Update.update(Update.java:102) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.dml.Merge.merge(Merge.java:154) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.dml.Merge.update(Merge.java:106) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.CommandContainer.update(CommandContainer.java:98) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.command.Command.executeUpdate(Command.java:258) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
... 7 common frames omitted
Caused by: java.nio.channels.ClosedChannelException: null
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110) ~[na:1.8.0_161]
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:721) ~[na:1.8.0_161]
at org.h2.store.fs.FileNio.read(FilePathNio.java:74) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:421) ~[h2-1.4.193.jar.4067930195476961885.tmp:1.4.193]
... 28 common frames omitted
20180731 10:01:40 [Scanner thread]

I think I found the issue.

After some research, what I did is the following:

Stop ubooquity

Delete both db file and the DB.old

Change the configuration file memory option from 512 to 4096

Start ubooquity

After a long scan I now have 30,000 books and comics available and without issues.

Thank you for trying to help me.

Adi

I'm glad you could solve your issue.

Out of curiosity, did you have some OutOfMemoryError messages in your logs when the problem was happening ?

not that i could see, however, if it appeared just once, most likely i missed it.