Amazon Simple Storage Service (S3)

Amazon S3 (Simple Storage Service) is a scalable, secure, and highly available object storage service that Amazon Web Services (AWS) provides. S3 allows you to store and retrieve any amount of data from anywhere on the internet. In this tutorial, we will explore the features of Amazon S3 and how to use it from both the AWS Management Console UI and the AWS CLI.

Features of Amazon S3

1. Scalability: S3 can scale to store and retrieve any amount of data, from a few gigabytes to exabytes of data.

2. Durability and Availability: S3 is designed to provide 99.999999999% durability and 99.99% availability. This means that your data is always available when you need it.

3. Security: S3 provides several security features, including encryption, access control, and compliance certifications such as PCI-DSS and HIPAA.

4. Lifecycle Policies: S3 allows you to define lifecycle policies for your objects, which automatically move data to cheaper storage classes or delete it when it is no longer needed.

5. Versioning: S3 enables versioning on your buckets, which allows you to store multiple versions of an object.

6. Cross-Region Replication: S3 allows you to replicate data to other AWS regions for disaster recovery or reduce latency.

Use cases

Amazon S3 is designed to store and retrieve any amount of data from anywhere on the web. Whether you need to store and protect data, host a static website, or build a big data analytics platform, S3 has the flexibility and features to meet your needs. Here are some common use cases for Amazon S3:

1. Data Backup and Archiving: S3 is an ideal solution for backing up and archiving data. You can store data in S3 as a cost-effective way to protect it from accidental deletion, hardware failure, and other potential threats.

2. Static Web Hosting: You can use S3 to host static websites, including HTML, CSS, and JavaScript files. You can also use S3 as a content delivery network (CDN) for static assets like images and videos.

3. Big Data Analytics: S3 can store large volumes of data that can be used for big data analytics. S3 can be a data lake for analytics tools like Amazon EMR, Amazon Athena, and Amazon Redshift.

4. Media Storage and Distribution: S3 is ideal for storing and distributing large media files, such as videos, music, and images. You can use S3 to deliver media files directly to your users through a CDN or streaming service like Amazon CloudFront.

5. Disaster Recovery: S3 can be used as a reliable disaster recovery solution. You can replicate data between S3 buckets in different AWS regions to ensure that your data is always available during an outage.

6. Mobile and IoT Application Data Storage: S3 is a perfect fit for storing data generated by mobile and IoT applications. Developers can use the AWS Mobile SDKs to access S3 and store data, such as photos, videos, and sensor data.

7. Content Management and Collaboration: S3 can be used as a central repository for content management and collaboration tools. Teams can use S3 to store and share files, collaborate on documents, and track revisions.

Basic usage

Using Amazon S3 from the AWS Management Console UI

Step 1: Create an S3 Bucket

To create a bucket, navigate to the AWS Management Console and S3. Click on "Create bucket" and enter a unique name for your bucket, select a region, and choose the appropriate settings for versioning and encryption.

Step 2: Upload Objects to Your Bucket

Once you have created your bucket, you can upload objects to it by clicking on "Upload" in the S3 console. You can drag and drop files or manually select them from your local machine.

Step 3: Set Permissions for Your Bucket

You can set permissions for your bucket by clicking on the "Permissions" tab in the S3 console. Here, you can set access policies, enable versioning, and configure encryption settings.

Using Amazon S3 from the AWS CLI

Step 1: Create an S3 Bucket

To create a bucket using the AWS CLI, use the following command:

aws s3api create-bucket --bucket my-bucket-name --region us-west-2 --create-bucket-configuration LocationConstraint=us-west-2

This command creates a bucket named "my-bucket-name" in the us-west-2 region.

Step 2: Upload Objects to Your Bucket

To upload objects to your bucket using the AWS CLI, use the following command:

aws s3 cp file.txt s3://my-bucket-name/

This command uploads the file "file.txt" to the bucket "my-bucket-name".

Step 3: Set Permissions for Your Bucket

To set permissions for your bucket using the AWS CLI, use the following command:

aws s3api put-bucket-acl --bucket my-bucket-name --acl public-read

This command sets the bucket ACL to "public-read", allowing anyone to read the bucket's objects.

Permissions: ACL vs Bucket Policy

Bucket policies and Access Control Lists (ACLs) are two mechanisms in Amazon S3 that can be used to control access to buckets and objects. While both can be used to grant or deny permissions, bucket policies offer some advantages over ACLs in certain scenarios:

1. Simplify Permissions Management: Bucket policies provide a simpler way to manage permissions for multiple users or roles. With bucket policies, you can define permissions for a group of users or roles in a single policy, whereas with ACLs, you must apply permissions to each object.

2. Granular Permissions: Bucket policies offer more granular control over permissions than ACLs. To restrict or allow access to buckets or objects, you can use bucket policies to define complex conditions, such as IP addresses, user agents, or time of day.

3. Cross-Account Access: Bucket policies enable cross-account access, which is impossible with ACLs. You can use bucket policies to grant access to users or roles in different AWS accounts.

4. Auditing and Logging: Bucket policies provide better auditing and logging capabilities than ACLs. You can use Amazon S3 server access logs to track access to buckets and objects and monitor changes to bucket policies using AWS CloudTrail.

5. Compliance Requirements: Some compliance requirements, such as HIPAA or PCI DSS, require the use of bucket policies to control access to data. If you must meet compliance requirements, bucket policies may be a better option than ACLs.

While ACLs can be useful in certain scenarios, such as granting permissions to individual objects or for simple access control requirements, bucket policies are generally more flexible and powerful. In most cases, it is recommended to use bucket policies instead of ACLs to control access to Amazon S3 buckets and objects.

Bucket policy examples

Here are some examples of bucket policies that demonstrate different ways you can use them to control access to S3 buckets:

Example 1: Granting public read access to all objects in a bucket

This policy grants public read access to all objects in a bucket, allowing anyone to view the bucket's contents

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"PublicReadGetObject",
         "Effect":"Allow",
         "Principal": "*",
         "Action":["s3:GetObject"],
         "Resource":["arn:aws:s3:::my-bucket/*"]
      }
   ]
}

Example 2: Restricting access to a specific IP address

This policy restricts access to objects in a bucket to a specific IP address or range of IP addresses.

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"AllowAccessFromSpecificIP",
         "Effect":"Allow",
         "Principal": "*",
         "Action":["s3:GetObject"],
         "Resource":["arn:aws:s3:::my-bucket/*"],
         "Condition":{
            "IpAddress":{
               "aws:SourceIp":"203.0.113.0/24"
            }
         }
      }
   ]
}

Example 3: Granting access to a specific IAM user or role

This policy grants access to objects in a bucket to a specific IAM user or role.

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"AllowAccessToUser",
         "Effect":"Allow",
         "Principal": {"AWS":"arn:aws:iam::123456789012:user/my-user"},
         "Action":["s3:GetObject"],
         "Resource":["arn:aws:s3:::my-bucket/*"]
      }
   ]
}

Practice

You need to host a simple static landing page using S3.

• Create a bucket in the us-east-1 (N. Virginia) region with the following settings:

  • ACLs are disabled (we will use the bucket policy instead).

  • Uncheck "block all public access", as our bucket must be publicly available.

  • Leave bucket versioning disabled and use default encryption settings.

• Upload the landing page files to it.

• Ensure that "Block Public Access" is turned off and set the bucket policy allowing read-only access from everywhere.

• Enable website hosting for this bucket and specify an index object path.

The website endpoint (URL) will be available in the "Static website hosting" section under the Properties tab.

After completing this task, please send your website URL to @purrplehaze.