1. General
  2. Bugs
0
Fixed

Error when scanning library

synobooks 9 years ago • updated by Tom 8 years ago • 9

I was trying to get Ubooquity to rebuild my library after upgrading to the new version 1.10 but it is unable to index beyond this book


Clearing the database and reindexing, it still gets stuck on this book.


20160510 01:58:51 [Scanner thread] INFO com.ubooquity.fileformat.pdf.b - Reading metadata of pdf file: /xxx/xxx/xxxx/xxxxx/x.pdf

20160510 01:58:52 [Scanner thread] ERROR com.ubooquity.Ubooquity - Uncaught exception on thread: Scanner thread

java.lang.NoClassDefFoundError: org/bouncycastle/jce/provider/BouncyCastleProvider
   at org.apache.pdfbox.pdmodel.encryption.PDEncryption.(PDEncryption.java:96) ~[pdfbox-2.0.0.jar.8584131639923899513.tmp:2.0.0]
   at org.apache.pdfbox.pdfparser.PDFParser.prepareDecryption(PDFParser.java:282) ~[pdfbox-2.0.0.jar.8584131639923899513.tmp:2.0.0]
   at org.apache.pdfbox.pdfparser.PDFParser.initialParse(PDFParser.java:199) ~[pdfbox-2.0.0.jar.8584131639923899513.tmp:2.0.0]
   at org.apache.pdfbox.pdfparser.PDFParser.parse(PDFParser.java:249) ~[pdfbox-2.0.0.jar.8584131639923899513.tmp:2.0.0]
   at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:847) ~[pdfbox-2.0.0.jar.8584131639923899513.tmp:2.0.0]
   at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:803) ~[pdfbox-2.0.0.jar.8584131639923899513.tmp:2.0.0]
   at org.apache.pdfbox.pdmodel.PDDocument.load(PDDocument.java:757) ~[pdfbox-2.0.0.jar.8584131639923899513.tmp:2.0.0]
   at com.ubooquity.fileformat.pdf.b.a(SourceFile:34) ~[Ubooquity.jar:1.10.0]
   at com.ubooquity.data.feeder.a.b(SourceFile:382) ~[Ubooquity.jar:1.10.0]
   at com.ubooquity.data.feeder.a.a(SourceFile:212) ~[Ubooquity.jar:1.10.0]
   at com.ubooquity.data.feeder.a.b(SourceFile:316) ~[Ubooquity.jar:1.10.0]
   at com.ubooquity.data.feeder.a.a(SourceFile:35) ~[Ubooquity.jar:1.10.0]
   at com.ubooquity.data.feeder.a$1.run(SourceFile:116) ~[Ubooquity.jar:1.10.0]
   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_92]
Caused by: java.lang.ClassNotFoundException: Failure to load: org.bouncycastle.jce.provider.BouncyCastleProvider
   at com.ubooquity.e.c.loadClass(SourceFile:992) ~[Ubooquity.jar:1.10.0]
   at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_92]
... 14 common frames omitted


Please assist.

synobooks 9 years ago

I went to look at the pdf file that it is getting stuck and realized the file is a (secured)pdf.


Perhaps one suggestion is to build in some error trapping or bypass so that the library can continue indexing other files instead of getting stuck once it encounters a secured pdf. =)

Planned
Tom 9 years ago

My bad, I'll fix that too in 1.10.1 (which will be done as soon as I have an hour to do it).

Fixed
Tom 9 years ago

Fixed in 1.10.1.

Batmanraid 8 years ago

I have 1.10.1 installed and i think the error persist.


This is the log:


20160622 16:02:55 [Scanner thread] INFO com.ubooquity.fileformat.pdf.b - Reading metadata of pdf file: /media/Biblioteca/Comics/Ballantine books/Tales of the Dark Knight (1989)/Tales of the Dark Knight (1989) #0001 (1989).pdf
20160622 16:03:33 [Scanner thread] ERROR com.ubooquity.Ubooquity - Uncaught exception on thread: Scanner thread
java.lang.OutOfMemoryError: Java heap space
   at java.awt.image.DataBufferByte.(DataBufferByte.java:92) ~[na:1.8.0_65]
   at java.awt.image.ComponentSampleModel.createDataBuffer(ComponentSampleModel.java:445) ~[na:1.8.0_65]
   at java.awt.image.Raster.createWritableRaster(Raster.java:941) ~[na:1.8.0_65]
   at javax.imageio.ImageTypeSpecifier.createBufferedImage(ImageTypeSpecifier.java:1074) ~[na:1.8.0_65]
   at javax.imageio.ImageReader.getDestination(ImageReader.java:2892) ~[na:1.8.0_65]
   at com.sun.imageio.plugins.jpeg.JPEGImageReader.readInternal(JPEGImageReader.java:1071) ~[na:1.8.0_65]
   at com.sun.imageio.plugins.jpeg.JPEGImageReader.read(JPEGImageReader.java:1039) ~[na:1.8.0_65]
   at com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageReader.read(Unknown Source) ~[imageio-jpeg-3.1.0.jar.9208094017101121644.tmp:3.1.0]
   at javax.imageio.ImageReader.read(ImageReader.java:939) ~[na:1.8.0_65]
   at org.apache.pdfbox.filter.DCTFilter.decode(DCTFilter.java:83) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.cos.COSInputStream.create(COSInputStream.java:69) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.cos.COSStream.createInputStream(COSStream.java:163) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.pdmodel.common.PDStream.createInputStream(PDStream.java:235) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.pdmodel.graphics.image.PDImageXObject.(PDImageXObject.java:147) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.pdmodel.graphics.PDXObject.createXObject(PDXObject.java:70) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.pdmodel.PDResources.getXObject(PDResources.java:385) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.contentstream.operator.graphics.DrawObject.process(DrawObject.java:53) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:815) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:472) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:446) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:149) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.rendering.PageDrawer.drawPage(PageDrawer.java:189) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.rendering.PDFRenderer.renderPage(PDFRenderer.java:208) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.rendering.PDFRenderer.renderImage(PDFRenderer.java:139) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at org.apache.pdfbox.rendering.PDFRenderer.renderImageWithDPI(PDFRenderer.java:94) ~[pdfbox-2.0.0.jar.4880181557651869286.tmp:2.0.0]
   at com.ubooquity.fileformat.pdf.b.a(SourceFile:71) ~[Ubooquity.jar:1.10.1]
   at com.ubooquity.fileformat.pdf.b.a(SourceFile:44) ~[Ubooquity.jar:1.10.1]
   at com.ubooquity.b.c.a(SourceFile:58) ~[Ubooquity.jar:1.10.1]
   at com.ubooquity.data.feeder.a.b(SourceFile:512) ~[Ubooquity.jar:1.10.1]
   at com.ubooquity.data.feeder.a.c(SourceFile:470) ~[Ubooquity.jar:1.10.1]
   at com.ubooquity.data.feeder.a.b(SourceFile:35) ~[Ubooquity.jar:1.10.1]

   at com.ubooquity.data.feeder.a$1.run(SourceFile:123) ~[Ubooquity.jar:1.10.1]

synobooks 8 years ago

It's a different error u are experiencing.


It's a OutOfMemoryError meaning u need to increase the amount of ram allocated to ubooquity. 1024mb works for me

boukaro007 8 years ago

Hello


I have the same problème with JRE8 and 1.10.1 with JRE7 et 1.9 no probléme


In the log, i have erreur string out line 18.


I have the problem with 12335 ebooks, under 10000 no probleme !


thanks for all

Tom 8 years ago

Yep, memory consumption is already on my todo list. I'll investigate.

benigngerm 8 years ago

I'm getting the out of memory error too on several pdf files. I tried increasing the Java heap size by adding the -Xmx1500m command to Java but that didn't do anything. 


20160727 18:01:36 [Scanner thread] ERROR com.ubooquity.Ubooquity - Uncaught exception on thread: Scanner thread
java.lang.OutOfMemoryError: Java heap space
    at java.awt.image.DataBufferInt.<init>(Unknown Source) ~[na:1.8.0_101]
    at java.awt.image.Raster.createPackedRaster(Unknown Source) ~[na:1.8.0_101]
    at java.awt.image.DirectColorModel.createCompatibleWritableRaster(Unknown Source) ~[na:1.8.0_101]
    at java.awt.image.BufferedImage.<init>(Unknown Source) ~[na:1.8.0_101]
    at sun.java2d.pipe.DrawImage.makeBufferedImage(Unknown Source) ~[na:1.8.0_101]
    at sun.java2d.pipe.DrawImage.renderImageXform(Unknown Source) ~[na:1.8.0_101]
    at sun.java2d.pipe.DrawImage.transformImage(Unknown Source) ~[na:1.8.0_101]
    at sun.java2d.pipe.DrawImage.transformImage(Unknown Source) ~[na:1.8.0_101]
    at sun.java2d.pipe.DrawImage.transformImage(Unknown Source) ~[na:1.8.0_101]
    at sun.java2d.SunGraphics2D.drawImage(Unknown Source) ~[na:1.8.0_101]
    at org.apache.pdfbox.rendering.PageDrawer.drawBufferedImage(PageDrawer.java:823) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.rendering.PageDrawer.drawImage(PageDrawer.java:781) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.contentstream.operator.graphics.DrawObject.process(DrawObject.java:62) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:815) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:472) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:446) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:149) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.rendering.PageDrawer.drawPage(PageDrawer.java:189) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.rendering.PDFRenderer.renderPage(PDFRenderer.java:208) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.rendering.PDFRenderer.renderImage(PDFRenderer.java:139) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at org.apache.pdfbox.rendering.PDFRenderer.renderImageWithDPI(PDFRenderer.java:94) ~[pdfbox-2.0.0.jar.8466586005085621666.tmp:2.0.0]
    at com.ubooquity.fileformat.pdf.b.a(SourceFile:71) ~[Ubooquity.jar:1.10.1]
    at com.ubooquity.fileformat.pdf.b.a(SourceFile:44) ~[Ubooquity.jar:1.10.1]
    at com.ubooquity.data.feeder.a.b(SourceFile:382) ~[Ubooquity.jar:1.10.1]
    at com.ubooquity.data.feeder.a.a(SourceFile:212) ~[Ubooquity.jar:1.10.1]
    at com.ubooquity.data.feeder.a.b(SourceFile:316) ~[Ubooquity.jar:1.10.1]
    at com.ubooquity.data.feeder.a.a(SourceFile:35) ~[Ubooquity.jar:1.10.1]
    at com.ubooquity.data.feeder.a$1.run(SourceFile:116) ~[Ubooquity.jar:1.10.1]
    at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_101]

Here is one of the pdf files if you need it for testing. https://www.dropbox.com/s/ffzl0yu9mhl01qt/Crimson%20Skies%20-%20Aircraft%20Manual.pdf?dl=0

Tom 8 years ago

Thanks for the sample file, I'll download it when I get home.

As for the solution, I am entirely dependent on the PDFBox library (used to extract PDF covers and pages).

So apart from updating Ubooquity with the latest PDFBox version each time I realease a new version, there's not much I can do.

Customer support service by UserEcho