Text To Speech, Highlight words as spoken

Technical Support Apryse SDK
1

Hi Everyone,

We’re building a TTS feature into our reader but I’ve encountered strange behavior while highlighting text as it is spoken.

I’m using the follow code to capture the word coordinates and render the highlight as the words are spoken.

for (TextExtractor.Line line = textExtractor.GetFirstLine(); line.IsValid(); line = line.GetNextLine())
{
for (TextExtractor.Word word = line.GetFirstWord(); word.IsValid(); word = word.GetNextWord())
{
var modifiedWord = word.GetString();
var box = word.GetQuad();
…

}
}

AND

var rect = new Rect();
rect.x1 = quads[0];
rect.y1 = quads[1];

rect.x2 = quads[2];
rect.y2 = quads[5];

Highlight = pdftron.PDF.Annots.Highlight.Create(_currentPdfDocument.GetSDFDoc(), Annot.Type.e_Underline, rect);
if (Highlight == null)
return;

Highlight.SetUniqueID(uniqueId);
Highlight.SetColor(new ColorPt(0, 0, 0), 3);

_currentPage.AnnotPushBack(Highlight);

PdfViewer.Update(rect);

As the text is speaking I am able to render most of the highlights. However when zoomed into the page, so that the entire page is not completely visible, it stops rendering my highlights at about 80% of the way down the page. When the entire page is visible, all highlights are rendered correctly. It happens in more than one book. I’ve validated that the coordinates are valid by selecting the highlight and comparing the results with the GetQuad() call above. It just simply stops rendering.

My counterpart working on the iOS client does not experience this issue.

Any thoughts or help you could provide would be very helpful!

1 Like
2

After many more hours of tweaks I’ve found that changing the last line from

PdfViewer.Update(rect);

to

PdfViewer.Update();

Fixes the issue. That said I have to think that this will have a negative impact on performance! Redrawing everything has to be worse than redrawing a simple rectangle.

1 Like
3

Hi Justin,

When passing the rect into the PDView.Update method, you must make sure the points are in screen coordinates and not page coordinates. To do so, you can use the following method to convert the point to screen coordinates:
https://www.pdftron.com/api/ios/pdftron/Classes/PTPDFViewCtrl.html#/c:objc(cs)PTPDFViewCtrl(im)ConvPagePtToScreenPt:page_num:

Please let me know if this works for you, and if you have any further questions.

1 Like
4

Sorry for the delay getting back to you. Your suggestion worked perfectly, thank you for the help!

1 Like
5

Greetings, am trying to develop a doc viewer in flutter but am having challenges to add custom icons buttons, they may be floating or on the toolbar

var prepMasterToolbar = CustomToolbar(“prepMaster”,“Prepmaster”,[IconButton(onPressed: (){}, icon: Icon(Icons.headphones)),IconButton(onPressed: (){}, icon: Icon(Icons.question_answer))],“Book”);
var annotationToolbars = [prepMasterToolbar, DefaultToolbars.annotate];
config.annotationToolbars = annotationToolbars;
PdftronFlutter.openDocument(slideFile.path, config: config);

the among the two custom icon buttons i want one to be for text to speech, i want when it is clicked, the tts engine must be initialized and start reading the current page whiles highlighting each word its reading.

any help.

1 Like