Add s3 protocol to `URIReader`
Created by: AlexKordic
Add S3 resource download to URIReader
class.
The S3 URLs look like:
s3+https://s3_key:secret@storage.googleapis.com/alexk-dms-upload-test/testvideo.ts
s3+https://s3_key:secret@s3.amazonaws.com/alexk-dms-upload-test/testvideo.ts
Where s3_key
and secret
fallback to environment variables S3_ACCESS_KEY_ID
and S3_SECRET_ACCESS_KEY
respectively.
We employ AWS
authorization method that should support google, amazon and other S3 compatible cloud providers.
This PR adds logic to calculate AWS
signature for S3 GET request. New logic is placed just before HTTP/HTTPS
protocols to create proper HTTP request for HTTP/HTTPS
logic. This means existing code path is reused and range requests are supported (supportRangeRequest == true
).
How to test
- Compile
- Run
./build/urireadertest "s3+https://your-key:your-secret@storage.googleapis.com/alexk-dms-upload-est/somedata.txt"
- Observe file contents is produced on
stdout