{"id":3247,"date":"2025-04-22T07:02:43","date_gmt":"2025-04-22T07:02:43","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/04\/22\/beginners-guide-to-creating-a-s3-storage-on-aws\/"},"modified":"2025-04-22T07:02:43","modified_gmt":"2025-04-22T07:02:43","slug":"beginners-guide-to-creating-a-s3-storage-on-aws","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/04\/22\/beginners-guide-to-creating-a-s3-storage-on-aws\/","title":{"rendered":"Beginner\u2019s Guide to Creating a S3 Storage on AWS"},"content":{"rendered":"<p>    Beginner\u2019s Guide to Creating a S3 Storage on AWS<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n    <!-- no image --><br \/>\n \t<BR><br \/>\n<BR><\/BR><\/p>\n<div>\n<h2 class=\"wp-block-heading\" id=\"3562\"><mdspan datatext=\"el1745296693599\" class=\"mdspan-comment\">Introduction<\/mdspan><\/h2>\n<p class=\"wp-block-paragraph\" id=\"db4c\"><a href=\"https:\/\/towardsdatascience.com\/tag\/aws\/\" title=\"AWS\">AWS<\/a> is a well-known cloud provider whose primary goal is to allocate server resources for software engineers to deploy their applications. AWS offers many services, one of which is EC2, providing virtual machines for running software applications in the cloud.<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\" id=\"fcc9\">However, for data-intensive applications, storing data inside EC2 instances is not always the optimal choice. While EC2 offers fast read and write speeds, it is not optimized for scalability. A better alternative is to use S3 storage instead.<\/p>\n<\/blockquote>\n<h2 class=\"wp-block-heading\" id=\"e2a5\">Storing data in EC2 vs S3<\/h2>\n<p class=\"wp-block-paragraph\" id=\"7a69\">Amazon <a href=\"https:\/\/towardsdatascience.com\/tag\/s3\/\" title=\"S3\">S3<\/a> was specifically designed for storing massive amounts of unstructured data:<\/p>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">It has a highly reliable resilience system, thanks to which the durability rate exceeds 99.99%.<\/li>\n<li class=\"wp-block-list-item\">S3 automatically replicates data across multiple servers to prevent potential data loss.<\/li>\n<li class=\"wp-block-list-item\">It seamlessly integrates with other AWS services for data analytics and machine learning.<\/li>\n<li class=\"wp-block-list-item\">Storing data in S3 is significantly more cost-effective compared to EC2.<\/li>\n<\/ul>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\" id=\"54f9\">The main use case where EC2 might be preferred is when frequent data access is required. For example, during machine learning model training, where the dataset must be read repeatedly for each batch. In most other cases, S3 is the better choice.<\/p>\n<\/blockquote>\n<h2 class=\"wp-block-heading\" id=\"69e4\">About this article<\/h2>\n<p class=\"wp-block-paragraph\" id=\"d42a\">The objective of this article is to demonstrate how to create a basic S3 <a href=\"https:\/\/towardsdatascience.com\/tag\/storage\/\" title=\"Storage\">Storage<\/a>. By the end of the tutorial, we will have a functioning S3 storage that allows remote access to uploaded images.<\/p>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\" id=\"8fbe\">To keep the focus on key aspects, we will cover only the storage creation process and not dive into best security practices.<\/p>\n<\/blockquote>\n<h2 class=\"wp-block-heading\" id=\"821a\">Tutorial<\/h2>\n<h3 class=\"wp-block-heading\" id=\"aff1\"># 01. Create S3 storage<\/h3>\n<p class=\"wp-block-paragraph\" id=\"0873\">To perform any operations related to S3 storage management, select the\u00a0<strong><em>Storage<\/em><\/strong>\u00a0option from the service menu. In the submenu that appears, choose\u00a0<strong><em>S3<\/em><\/strong>.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"460\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-91-1024x460.png?resize=1024%2C460&#038;ssl=1\" alt=\"\" class=\"wp-image-601953\"><\/figure>\n<p class=\"wp-block-paragraph\">AWS organizes data into collections called\u00a0<strong>buckets<\/strong>. To create a bucket, click\u00a0<strong><em>Create bucket<\/em><\/strong>.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"464\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-92-1024x464.png?resize=1024%2C464&#038;ssl=1\" alt=\"\" class=\"wp-image-601954\"><\/figure>\n<p class=\"wp-block-paragraph\">Each bucket requires a unique global name. Most other settings can be left as default.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"465\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-93-1024x465.png?resize=1024%2C465&#038;ssl=1\" alt=\"\" class=\"wp-image-601955\"><\/figure>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"444\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-94-1024x444.png?resize=1024%2C444&#038;ssl=1\" alt=\"\" class=\"wp-image-601956\"><\/figure>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"467\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-95-1024x467.png?resize=1024%2C467&#038;ssl=1\" alt=\"\" class=\"wp-image-601957\"><\/figure>\n<p class=\"wp-block-paragraph\" id=\"b505\">Once all options are selected, click\u00a0<strong><em>Create bucket<\/em><\/strong>. After a few seconds, AWS will redirect you to the bucket management panel.<\/p>\n<h3 class=\"wp-block-heading\" id=\"9336\"># 02. Create folder (optional step)<\/h3>\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\" id=\"76d1\">Folders in S3 function similarly to standard computer folders, helping to organize hierarchical data. Additionally, any file stored in an S3 folder will have a URL prefix that includes the folder path.<\/p>\n<\/blockquote>\n<p class=\"wp-block-paragraph\" id=\"ccaa\">To create a folder, click the\u00a0<strong><em>Create folder<\/em><\/strong>\u00a0button.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"309\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-96-1024x309.png?resize=1024%2C309&#038;ssl=1\" alt=\"\" class=\"wp-image-601958\"><\/figure>\n<p class=\"wp-block-paragraph\">In the appearing window, choose a custom name for the folder.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"492\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-97-1024x492.png?resize=1024%2C492&#038;ssl=1\" alt=\"\" class=\"wp-image-601959\"><\/figure>\n<p class=\"wp-block-paragraph\">After clicking the\u00a0<strong><em>Create folder button<\/em><\/strong>, the folder will be created! You can now navigate to it. Since no images have been uploaded yet, the folder is empty for now, but we will add images in step 4.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"288\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-98-1024x288.png?resize=1024%2C288&#038;ssl=1\" alt=\"\" class=\"wp-image-601960\"><\/figure>\n<h3 class=\"wp-block-heading\" id=\"38a5\"># 03. Adjust data access<\/h3>\n<p class=\"wp-block-paragraph\" id=\"727e\">As a reminder, our goal is to create a publicly visible image storage that allows remote access. To achieve this, we need to adjust data access policies.<\/p>\n<p class=\"wp-block-paragraph\" id=\"7aa0\">By clicking on the\u00a0<strong><em>Permissions<\/em><\/strong>\u00a0tab under the bucket name, you will see a list of options to modify access settings.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"513\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-99-1024x513.png?resize=1024%2C513&#038;ssl=1\" alt=\"\" class=\"wp-image-601961\"><\/figure>\n<p class=\"wp-block-paragraph\">We need to unblock public access, so click on the respective\u00a0<strong><em>Edit button<\/em><\/strong>\u00a0in the interface and uncheck all the checkboxes related to access blocking.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"433\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-100-1024x433.png?resize=1024%2C433&#038;ssl=1\" alt=\"\" class=\"wp-image-601962\"><\/figure>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"424\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-101-1024x424.png?resize=1024%2C424&#038;ssl=1\" alt=\"\" class=\"wp-image-601963\"><\/figure>\n<p class=\"wp-block-paragraph\" id=\"8531\">After saving the changes, we should see an exclamation mark icon with the\u00a0<em>\u201cOff\u201d<\/em>\u00a0text. Then, navigate to the\u00a0<strong><em>Bucket policy\u00a0<\/em><\/strong>section and click\u00a0<strong><em>Edit<\/em><\/strong>.<\/p>\n<p class=\"wp-block-paragraph\" id=\"2924\">To allow read access, insert the following policy text:<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"474\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-102-1024x474.png?resize=1024%2C474&#038;ssl=1\" alt=\"\" class=\"wp-image-601964\"><\/figure>\n<h3 class=\"wp-block-heading\" id=\"8b13\"># 04. Upload images<\/h3>\n<p class=\"wp-block-paragraph\" id=\"7fcf\">Now it is time to upload images. To do that, navigate to the created\u00a0<em>\u201cimages\u201d<\/em> folder and click on the\u00a0<strong><em>Upload<\/em><\/strong>\u00a0button.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"351\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-103-1024x351.png?resize=1024%2C351&#038;ssl=1\" alt=\"\" class=\"wp-image-601965\"><\/figure>\n<p class=\"wp-block-paragraph\">Click on the\u00a0<strong><em>Add files<\/em><\/strong>\u00a0button, which will open a file explorer on your computer. Choose and import the images from there.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"276\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-104-1024x276.png?resize=1024%2C276&#038;ssl=1\" alt=\"\" class=\"wp-image-601966\"><\/figure>\n<p class=\"wp-block-paragraph\">Depending on the number and size of the imported images, AWS might take some time to process them.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"411\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-105-1024x411.png?resize=1024%2C411&#038;ssl=1\" alt=\"\" class=\"wp-image-601967\"><\/figure>\n<p class=\"wp-block-paragraph\">In this example, I have imported nine images.<\/p>\n<h3 class=\"wp-block-heading\" id=\"5cd7\"># 05. Access data<\/h3>\n<p class=\"wp-block-paragraph\" id=\"842e\">After the images have been successfully imported, click on any of their filenames to get more information.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"410\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-106-1024x410.png?resize=1024%2C410&#038;ssl=1\" alt=\"\" class=\"wp-image-601970\"><\/figure>\n<p class=\"wp-block-paragraph\" id=\"d646\">In the opened panel, you will see metadata related to the chosen image. As we can see in the\u00a0<em>\u201cObject URL\u201d<\/em>\u00a0field, AWS created a unique URL for our image!<\/p>\n<p class=\"wp-block-paragraph\" id=\"9108\">Additionally, we can notice that the URL contains the images\/ prefix, which corresponds exactly to the folder structure we defined above!<\/p>\n<p class=\"wp-block-paragraph\" id=\"ce13\">Finally, since we have authorized read access, we can now publicly access this URL.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"412\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-107-1024x412.png?resize=1024%2C412&#038;ssl=1\" alt=\"\" class=\"wp-image-601971\"><\/figure>\n<p class=\"wp-block-paragraph\">If you click on the image URL and copy it into the browser\u2019s address bar, the image will be displayed!<\/p>\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" height=\"516\" width=\"1024\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/contributor.insightmediagroup.io\/wp-content\/uploads\/2025\/04\/image-108-1024x516.png?resize=1024%2C516&#038;ssl=1\" alt=\"\" class=\"wp-image-601972\"><\/figure>\n<p class=\"wp-block-paragraph\" id=\"8cfe\">The amazing part about this is that you can now create a URL template in the form\u00a0<code>https:\/\/&lt;bucket_url&gt;\/&lt;folder_path&gt;\/&lt;filename&gt;<\/code>.<\/p>\n<p class=\"wp-block-paragraph\" id=\"1ff3\">By doing so, you can dynamically replace the &lt;filename&gt; field in a program to access images and perform data manipulation.<\/p>\n<h2 class=\"wp-block-heading\" id=\"6ca9\">Conclusion<\/h2>\n<p class=\"wp-block-paragraph\" id=\"f747\">In this article, we have introduced the AWS S3 storage system, which is very useful for storing large amounts of unstructured data. With its advanced scalability and security mechanisms, S3 is perfect for organizing massive data volumes at a much lower cost compared to EC2 containers.<\/p>\n<p class=\"wp-block-paragraph\" id=\"1e82\"><em>All images are by the author unless noted otherwise.<\/em><\/p>\n<h2 class=\"wp-block-heading\">Connect with me<\/h2>\n<ul class=\"wp-block-list\">\n<li class=\"wp-block-list-item\">\n<strong><a href=\"https:\/\/medium.com\/@slavahead\" data-type=\"link\" data-id=\"https:\/\/medium.com\/@slavahead\">Medium<\/a><\/strong> <img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/270d.png?ssl=1\" alt=\"\u270d\" class=\"wp-smiley\" style=\"height: 1em; max-height: 1em;\">\n<\/li>\n<li class=\"wp-block-list-item\">\n<strong><a href=\"https:\/\/www.linkedin.com\/in\/vyacheslav-efimov\/\" data-type=\"link\" data-id=\"https:\/\/www.linkedin.com\/in\/vyacheslav-efimov\/\">LinkedIn<\/a><\/strong> <img data-recalc-dims=\"1\" decoding=\"async\" src=\"https:\/\/i0.wp.com\/s.w.org\/images\/core\/emoji\/15.0.3\/72x72\/1f9d1-200d-1f4bb.png?ssl=1\" alt=\"\ud83e\uddd1\u200d\ud83d\udcbb\" class=\"wp-smiley\" style=\"height: 1em; max-height: 1em;\">\n<\/li>\n<\/ul>\n<p>The post <a href=\"https:\/\/towardsdatascience.com\/beginners-guide-to-creating-a-s3-storage-on-aws\/\">Beginner\u2019s Guide to Creating a S3 Storage on AWS<\/a> appeared first on <a href=\"https:\/\/towardsdatascience.com\/\">Towards Data Science<\/a>.<\/p>\n<\/div>\n<p> \t<BR><br \/>\n <BR><\/BR><br \/>\n    Vyacheslav Efimov<br \/>\n \t<BR><br \/>\n<BR><\/BR><br \/>\n<a href=\"https:\/\/towardsdatascience.com\/beginners-guide-to-creating-a-s3-storage-on-aws\/\">Go to original source<\/a><br \/>\n \t<BR><br \/>\n <BR><\/BR><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Beginner\u2019s Guide to Creating a S3 Storage on AWS Introduction AWS is a well-known cloud provider whose primary goal is to allocate server resources for software engineers to deploy their applications. AWS offers many services, one of which is EC2, providing virtual machines for running software applications in the cloud. However, for data-intensive applications, storing [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[62,2418,2419,401,938,2420,2421],"tags":[84,868,2422],"class_list":["post-3247","post","type-post","status-publish","format-standard","hentry","category-aimldsaimlds","category-aws","category-cloud","category-data-engineering","category-data-processing","category-s3","category-storage","tag-data","tag-s","tag-storage"],"_links":{"self":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/3247"}],"collection":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/comments?post=3247"}],"version-history":[{"count":0,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/posts\/3247\/revisions"}],"wp:attachment":[{"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/media?parent=3247"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/categories?post=3247"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mailitics.com\/index.php\/wp-json\/wp\/v2\/tags?post=3247"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}