V3.0.1 beta scans not showing complete on dashboard but logs show complete
When a scan is started it adds all the new items found and the user can view/ download etc but on the admin dashboard it does not show as complete the Total line is not populated. The log file though indicates that it is complete or at least it does from what I can tell. When I try to start another scan I get the message that one is already running. At the moment I simply stop and restart the server to resolve this. When I restart it then shows the total line populated from the previous scan
Here are the last lines from my log file after I stopped and restarted the server:
20231105 08:36:25 [Scanner thread] INFO com.ubooquity.a.a.a.d - Inserting/updating opus entry in database : L:\Magazines\Crafts
20231105 08:36:26 [Scanner thread] INFO com.ubooquity.a.a.f - Found 14800 empty folders
20231105 08:36:28 [Scanner thread] INFO com.ubooquity.a.a.f - Deleted empty folders from database. Looking for new ones...
20231105 08:36:28 [Scanner thread] INFO com.ubooquity.a.a.f - Found 144 empty folders
20231105 08:36:29 [Scanner thread] INFO com.ubooquity.a.a.f - Deleted empty folders from database. Looking for new ones...
20231105 08:36:30 [Scanner thread] INFO com.ubooquity.a.b.a - Finished scanning directories. Populating caches...
20231105 08:36:35 [Scanner thread] INFO com.ubooquity.a.b.a - Caches populated
20231105 08:36:35 [Scanner thread] INFO com.ubooquity.a.b.a - Statistics: [total: 148999, added or updated: 8, removed: 0]
20231105 13:55:28 [AWT-EventQueue-0] INFO com.ubooquity.c.f - Applying preferences modifications and restarting server
20231105 13:55:28 [AWT-EventQueue-0] INFO com.ubooquity.a - Stopping internal server...
20231105 13:55:28 [AWT-EventQueue-0] INFO com.ubooquity.a.b.a - Aborting content scan...
20231105 13:55:28 [AWT-EventQueue-0] INFO com.ubooquity.a.b.a - Waiting for scanner thread to stop
20231105 13:55:33 [AWT-EventQueue-0] INFO com.ubooquity.a.b.a - Interrupting stuck scanner thread
20231105 13:55:33 [AWT-EventQueue-0] INFO com.ubooquity.a.b.a - Scanner thread stopped
20231105 13:55:33 [AWT-EventQueue-0] INFO org.eclipse.jetty.server.Server - Stopped Server@9a20cbd{STOPPING}[11.0.15,sto=0]
20231105 13:55:33 [AWT-EventQueue-0] INFO o.e.jetty.server.AbstractConnector - Stopped library@7126e26{HTTP/1.1, (http/1.1)}{0.0.0.0:2202}
20231105 13:55:33 [AWT-EventQueue-0] INFO o.e.jetty.server.AbstractConnector - Stopped admin@35adf623{HTTP/1.1, (http/1.1)}{0.0.0.0:2203}
20231105 13:55:33 [AWT-EventQueue-0] INFO o.e.j.server.handler.ContextHandler - Stopped o.e.j.s.h.ContextHandler@3bb9ca38{/,null,STOPPED,@library}
20231105 13:55:33 [AWT-EventQueue-0] INFO o.e.j.server.handler.ContextHandler - Stopped o.e.j.s.h.ContextHandler@5934153e{/,null,STOPPED,@admin}
20231105 13:55:33 [AWT-EventQueue-0] INFO com.ubooquity.a - Internal server stopped
20231105 13:55:33 [AWT-EventQueue-0] INFO c.u.pref.PreferencesRepository - Saving preferences...
20231105 13:55:33 [AWT-EventQueue-0] INFO com.ubooquity.a - Starting internal server...
20231105 13:55:33 [AWT-EventQueue-0] INFO com.ubooquity.a.b.a - Ignoring scan request as another scan is already running.
20231105 13:55:33 [Scanner thread] ERROR com.ubooquity.Ubooquity - Uncaught exception on thread: Scanner thread
org.h2.jdbc.JdbcSQLNonTransientException: General error: "org.h2.mvstore.MVStoreException: Reading from file sun.nio.ch.FileChannelImpl@9f60d0c failed at 189418556 (length -1), read 0, remaining 1536 [2.2.222/1]"; SQL statement:
select o.filepath from opus o
join (select filename, filesize, count(*) from opus group by filename, filesize having count(*) > 1 ) b
on o.filename = b.filename and o.filesize = b.filesize
order by o.filename [50000-222]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:566)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:489)
at org.h2.message.DbException.get(DbException.java:212)
at org.h2.message.DbException.convert(DbException.java:407)
at org.h2.command.Command.executeQuery(Command.java:213)
at org.h2.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:131)
at com.ubooquity.a.a.a.a.c(SourceFile:596)
at com.ubooquity.a.a.e.f(SourceFile:156)
at com.ubooquity.a.b.a.h(SourceFile:116)
at com.ubooquity.a.b.a.b(SourceFile:102)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.h2.mvstore.MVStoreException: Reading from file sun.nio.ch.FileChannelImpl@9f60d0c failed at 189418556 (length -1), read 0, remaining 1536 [2.2.222/1]
at org.h2.mvstore.DataUtils.newMVStoreException(DataUtils.java:996)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:455)
at org.h2.mvstore.FileStore.readFully(FileStore.java:719)
at org.h2.mvstore.SingleFileStore.readFully(SingleFileStore.java:60)
at org.h2.mvstore.SingleFileStore.readFully(SingleFileStore.java:28)
at org.h2.mvstore.SFChunk.readFully(SFChunk.java:35)
at org.h2.mvstore.Chunk.readBufferForPage(Chunk.java:465)
at org.h2.mvstore.FileStore.readPage(FileStore.java:1960)
at org.h2.mvstore.MVStore.readPage(MVStore.java:1021)
at org.h2.mvstore.MVMap.readPage(MVMap.java:632)
at org.h2.mvstore.Page$NonLeaf.getChildPage(Page.java:1117)
at org.h2.mvstore.Cursor.hasNext(Cursor.java:64)
at org.h2.mvstore.tx.TransactionMap$CommittedIterator.fetchNext(TransactionMap.java:960)
at org.h2.mvstore.db.MVPrimaryIndex$MVStoreCursor.next(MVPrimaryIndex.java:440)
at org.h2.index.IndexCursor.next(IndexCursor.java:292)
at org.h2.table.TableFilter.next(TableFilter.java:425)
at org.h2.command.query.Select.gatherGroup(Select.java:517)
at org.h2.command.query.Select.queryGroup(Select.java:488)
at org.h2.command.query.Select.queryWithoutCache(Select.java:828)
at org.h2.command.query.Query.queryWithoutCacheLazyCheck(Query.java:197)
at org.h2.command.query.Query.query(Query.java:520)
at org.h2.command.query.Query.query(Query.java:483)
at org.h2.index.QueryExpressionIndex.find(QueryExpressionIndex.java:269)
at org.h2.index.QueryExpressionIndex.find(QueryExpressionIndex.java:152)
at org.h2.index.IndexCursor.find(IndexCursor.java:166)
at org.h2.table.TableFilter.next(TableFilter.java:394)
at org.h2.table.TableFilter.next(TableFilter.java:464)
at org.h2.command.query.Select$LazyResultQueryFlat.fetchNextRow(Select.java:1843)
at org.h2.result.LazyResult.hasNext(LazyResult.java:78)
at org.h2.result.FetchedResult.next(FetchedResult.java:34)
at org.h2.command.query.Select.queryFlat(Select.java:728)
at org.h2.command.query.Select.queryWithoutCache(Select.java:833)
at org.h2.command.query.Query.queryWithoutCacheLazyCheck(Query.java:197)
at org.h2.command.query.Query.query(Query.java:520)
at org.h2.command.query.Query.query(Query.java:483)
at org.h2.command.CommandContainer.query(CommandContainer.java:252)
at org.h2.command.Command.executeQuery(Command.java:192)
... 6 common frames omitted
Caused by: java.nio.channels.ClosedByInterruptException: null
at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:200)
at java.base/sun.nio.ch.FileChannelImpl.endBlocking(FileChannelImpl.java:172)
at java.base/sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:992)
at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:964)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:441)
... 41 common frames omitted
20231105 13:55:33 [AWT-EventQueue-0] INFO com.ubooquity.a - Reverse proxy prefix: collection
Here is my system info:
Ubooquity version: | 3.0.1 built on 2023-10-15 at 17:52 |
Java version: | 21.0.1 |
Java vendor: | Oracle Corporation |
Java VM name: | Java HotSpot(TM) 64-Bit Server VM |
OS name: | Windows 11 |
OS version: | 10.0 |
OS architecture: | amd64 |
Number of processors: | 16 |
Max memory: | 8176 MB |
Free memory: | 216 MB |
Total memory: | 708 MB |
Let me know if there is anything else you need or changes that I should make to my setup to get it working.
Customer support service by UserEcho
This one's harder. Working on it...
Thanks let me know if you need anymore details or anything you would like me to try
Since 3.0.0, Ubooquity looks for duplicate files after scanning collections.
This can be a veeeeery long operation on big collections and cause issues like the one you encountered, so I deactivated it by default and added an option (in advanced settings) to re-enable it.
Here is the fixed version (3.0.2) : https://vaemendis.net/ubooquity/downloads/Ubooquity-3.0.2.zip
Let me know how it goes.
My scans show as complete now and I can launch another scan as well. On my library previously it never showed complete and after anywhere between 24-48 hrs after I started a scan the app would crash or my pc would lock up if I didn't stop the server and start again. Did you make any changes to the duplicate detection that you would like me to try out and see if there are any improvements apart from the new flag?
Did you make any changes to the duplicate detection that you would like
me to try out and see if there are any improvements apart from the new
flag?
Thanks, for now I just made the feature optional.
I'll look into optimizations after the release of 3.1.0 (which will be the official Ubooquity 3, not in beta anymore).