Question:
We are trying to stream a PDF from AWS S3, convert to XOD, and then stream back up to S3. However, we are getting quite long times for this. In particular for a 10MB PDF, it is taking ~110 seconds on quad core i7/16gb machine. Is this expected?
Answer:
I converted that file (no downloading/uploading) and I got ~11.5 seconds, which is expected. I was running i7-3820 @ 3.6GHz with 16GB RAM.
First, note that you cannot upload to S3 until you know the file (contents) size, so you would need to finish conversion to XOD first. Though you could, during conversion, stream directly the viewer, and then store on S3 once conversion is done.
For testing, I would eliminate the conversion step, and just test your streaming. That is, stream down from one url, and just copy the bytes back up to a different URL. How long does that take? I image that is going to be ~100 seconds (for the PDF you sent me before).
Also, is this setup reflective of your final solution. If your code is going to run on a server somewhere, with a better internet connection, then you have in your own testing environment, than I would not do non-functional testing on your computer. Also, your server might have a higher upload throughput (typically upload is a lot slower than download rates).
It is possible, using an AWS SDK (there are a few options out there) to do multi-part uploading. That might help. Again, I believe you need to wait until the entire XOD is available, so you know the file size, and then you could do multiple uploads at once. This certainly would help with large files, say 100+MB, but for smaller files you would see decreasing return.
http://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html