Skip to content

Code library that provides a Stream implementation that makes working with uploads to S3 easier where the size of the content isn't known a priori. It holds only partial content in memory (works with large objects), and is compatible with code libraries that work with output streams.

License

Notifications You must be signed in to change notification settings

mlhpdx/s3-upload-stream

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

S3 Upload Stream

This library performs uploads to S3 without holding the entire object content in memory (or storage) beforehand. This approach is more efficient than using MemoryStream and offers compatibility with libraries and packages that work with a Stream interface. Examples for extracting GZIP, decrypting AES and PGP, and uploading dynamically-generated CSV are provided.

This project is a follow-on to SeekableS3Stream, also on Github. Using both libraries allows for very efficient, and simple, large-file processing to and from S3 via AWS Lambda, as demonstrated in one of the Examples.

For a more detailed explaination, check out the article on Medium.

About

Code library that provides a Stream implementation that makes working with uploads to S3 easier where the size of the content isn't known a priori. It holds only partial content in memory (works with large objects), and is compatible with code libraries that work with output streams.

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors 4

  •  
  •  
  •  
  •  

Languages