Chloe McAree (McAteer)
Published on

Securely Accessing Private Files Within Your Client

Authors

Accessing Private Files

When building media rich applications, that handle assets like images, audio or video — some of these assets may be publicly available, while some might be stored privately and only accessible by specific users.

If you are familiar with AWS, you may have come across AWS S3. Which is Amazon’s Simple Storage Service for object storage, that offers data availability, security, and performance at scale.

In S3, files are referred to as objects and by default, all objects are private, meaning only the owner of the object has permission to access it.

To work with AWS services like S3 in backend server-side applications, it is common to use the AWS SDK (Software Development Kit), which is available in many popular languages.

There are a range of AWS SDK’s available for different languagesThere are a range of AWS SDK’s available for different languages

Using the AWS SDK within the backend server-side code is a secure and effective approach to programmatically interact with AWS services, as you can authenticate with the SDK using your AWS client ID and client secret and access the required service functionalities.

However, when it comes to client-side applications like web or mobile apps, this option is not recommended.

These applications are typically bundled and publicly available. Since they are public, it allows people to inspect the code or app bundle, exposing the secret AWS credentials and posing a security risk. Unauthorised access to these credentials could potentially allow someone to gain access to all our AWS infrastructure.

You might be thinking, why don’t we just send a request from the client slide to the server and then within the server download the file from s3 using the SDK and send it back to the client?

Well you can do this, but there are several drawbacks to consider. Firstly, depending on the file sizes, sending them in the request can lead to excessive data transfers and performance issues.

Secondly, you would be serving images directly from the server, utilising valuable compute resources. If we were to offload this to the data source (S3) instead, you could optimise the server’s performance and resource allocation.

You would also want to avoid the need for multiple long-lasting connections to the server to hopefully enhance the scalability and efficiency of the server.

Additionally, if your are using API Gateway which is AWS’ API management tool that sits between a client and a collection of backend services, it imposes a 30-second timeout restriction. This may not provide enough time for a complete file transfer, complicating the server-to-client transmission.

In order to avoid these issues, we can look at using pre-signed urls and pre-signed cookies.

Pre-Signed URLs and Pre-signed Cookies

AWS pre-signed URLs and pre-signed cookies are authentication mechanisms provided by AWS that allow you to grant temporary access to your content to a limited number of users without exposing your AWS credentials.

Pre-signed Urls

Pre-signed URLs grant temporary access to an S3 object and are signed with AWS security credentials to authenticate access to a resource for a specific duration of time.

This allows users or clients who possess the pre-signed URL to access the object directly, without requiring their own AWS credentials.

Now, let’s explore how pre-signed URLs work in practice, with a real world example.

Within one of the applications I am currently working on at Hamilton Robson we need to store users passport images. Users should be able to view their own passport within their profile.

Let’s walk through the flow, starting with the client, which in this case is a mobile app (though the flow remains the same for web apps).

  1. The user navigates to their profile page and clicks to view their passport, triggering the client to send a request to API Gateway, a service which sits between the client and backend services.

  2. Here I have a custom authoriser set up to verify the users e.g. check their Bearer token to see if they are logged in with a valid session. If the user is authenticated and authorised, they proceed to the backend service.

  3. The backend service could be implemented using various AWS services, such as AWS Lambda or an EC2 instance. Here we would look up database to get the filename/path of this users passport image. Within this service we can authenticate with our AWS credentials, enabling us to utilise the AWS SDK and its functions to generate a pre-signed URL specific to the user and the desired file, with an expiry of 60 seconds.

  4. Once the pre-signed URL is generated, it is sent back to the client.

  5. The client can then use this authenticated URL to directly access the passport stored in S3 without requiring any further authentication, until the url expires.

This way, the user can securely access the passport image from S3 without directly exposing any AWS credentials or granting long-term access permissions.

This flow can be used not only for fetching assets but also for uploading assets. In the case of uploading, the pre-signed URL enables the client to directly upload the file to the S3 bucket using the provided URL, eliminating the need to pass the file through the backend service.

Generating pre-signed urls

    import aws from 'aws-sdk';
    const s3Client = new aws.S3({
      signatureVersion: 'v4',
      region: 'eu-west-2',
    });

    const url = await s3Client.getSignedUrl('getObject', {
      Bucket: BUCKET_NAME,
      Key: filePath,
      Expires: 90,
    });

In the code snippet above, I am using the SDK’s getSignedUrl function and passing in the S3 bucket name, the file path of the resource we want to access, and expiry time.

When using pre-signed URLs, S3 checks the expiration date and time of the signed URL at the time of the HTTP request. For example, if a client starts downloading a large file right before the expiration time, the download should still complete even if the expiration time passes during the download process. However, if the connection drops and the client attempts to restart the download after the expiration time has passed, the download will fail.

Benefits/ drawbacks of pre-signed urls

The main benefit of pre-signed URLs is the ability to enhance security by limiting access to authorised users and reducing the need for additional authentication mechanisms. They also provide more fine-grained control over access to S3 objects.

However, pre-signed URLs do have some limitations. One of the main limitations is that they can only be used to grant access or upload a single file. In other words, 1 URL = 1 file!

Pre-signed cookies

AWS pre-signed cookies are a way to grant temporary access to S3 objects. They are signed with AWS security credentials and stored in a cookie that can be passed along with HTTP requests to access the protected content.

To generate a pre-signed cookie, you need to set up a few things that are not necessarily required when using pre-signed URLs. Firstly, you must create a CloudFront distribution. Then, you need to generate a private/public key pair, where the public key will be uploaded to CloudFront and associated with the distribution.

To better understand the overall process, let’s walk through how pre-signed cookies work in practice, with a real world example.

In Hamilton Robson we have built a number of applications for large visitor attractions, all of which utilise the same underlying API.

Within our visitor attraction apps, there is some free content that is publicly accessible, but there is also paid content that can only be accessed by ticket holders.

  1. The user, within the client application, requests to verify their ticket, which triggers the app send a request to our backend server.

  2. Similar to the previous flow, we utilise API Gateway, which is equipped with a custom authoriser to verify the user’s ticket credentials.

  3. After verifying the ticket and identifying the attraction, we leverage the AWS SDK to generate a pre-signed cookie to all resources we store for that attraction using our private key. The details of how this process works will be explained shortly.

  4. Once the pre-signed cookie is generated, it is sent back to the client.

  5. The client can then use the cookie to download all the resources related to this attraction. These resources may include artwork, audio files for the audio guide, and various videos.

  6. When the client makes the request using the cookie, CloudFront, validates the signature of the cookie using the associated public key and checks the request’s policy before granting access to the requested assets. The cookie generated has a specified time limit, indicating how long it remains valid for before the user needs to re-authenticate.

Generating pre-signed cookies

The following code snippet shows how we can generate a pre-signed cookie using the SDK.

    const AWS = require("aws-sdk");

    const signer = new AWS.CloudFront.Signer(process.env.KEY_PAIR_ID, process.env.PRIVATE_KEY);

    const options = {
      url:  `${process.env.CLOUDFRONT_URL}/${attraction_id}/*`;,
      expires: Math.round(new Date().getTime() / 1000) + 3600 //unix UTC timestamp
      policy, //A CloudFront JSON policy
    };

    const cookieData = await signer.getSignedCookie(options);

Previously, I mentioned that when using pre-signed cookies, public and private keys are required. During the configuration of CloudFront, you provide your public key and receive a Key Pair ID. In the code snippet above, you can see I pass the Key Pair ID along with our private key to create a signer.

Next, we set up the function options. In the example, you can see the URL field uses the CloudFront base URL, and here we specify the path to the resources for which we want to generate the cookie. It’s worth noting that wildcards can be used to provide access to entire directories. This means that there’s no need to generate an individual cookie for each resource.

Additionally, we pass in a Unix timestamp to indicate the cookie’s expiration time. Once the cookie expires, the user will need to re-authenticate to gain access again.

Benefits/ drawbacks of pre-signed cookies

Some of the benefits of pre-signed cookies include their ease of use, as browsers automatically send the cookie with each request. Pre-signed cookies can also have longer lifetimes compared to pre-signed URLs. Another significant advantage is that pre-signed cookies can be used to access multiple objects in S3, offering a more efficient and streamlined approach.

However, it’s important to note that pre-signed cookies have certain limitations. They are more complex to set up as they require the generation of public and private keys, as well as the configuration of a CloudFront distribution.

Differences between pre-signed urls and pre-signed cookies

Overall, pre-signed URLs and signed cookies provide the same basic functionality of allowing you to control who can access your content. The choice between which one to use will depend on the specific use case and requirements of your application:

Pre-signed URLs are preferable when you need to share private content with a limited number of users or when you want to grant temporary access to content to users without AWS credentials. On the other hand, pre-signed cookies are more suitable when you need to grant temporary access to multiple S3 objects.

It’s worth mentioning that signed URLs take precedence over signed cookies. If both signed URLs and signed cookies are used to control access to the same files, and a viewer requests a file using a signed URL, CloudFront determines whether to return the file to the viewer based solely on the signed URL.

Best Practices

To enhance the security and effectiveness of using pre-signed URLs and cookies, here are some recommended best practices:

  1. Set short expiration times on URLs or cookies: By setting shorter expiration times, you minimise the risk of unauthorised access and enforce re-authentication if needed.

  2. Always use HTTPS: Transmit pre-signed URLs and cookies over HTTPS to ensure the encryption of data and prevent interception or tampering.

  3. Generate pre-signed URLs or cookies only for authorised users: Ensure that you generate pre-signed URLs or cookies only for users who have proper authorisation to access the content. This helps to mitigate the risk of unauthorised access and maintain data integrity.

Following these best practices can help ensure the security and integrity of your application.

Conclusion

Overall, AWS pre-signed URLs and pre-signed cookies are powerful tools that can improve the security and performance of your web applications. Pre-signed URLs and cookies allow you to grant temporary access to content without requiring AWS credentials or exposing sensitive information to unauthorised users.

While this blog post focused on the use of pre-signed URLs and cookies for resource fetching and uploading, it’s important to note that these mechanisms can also be utilised within content hosting for restricting web page access. However, delving into those aspects would require a separate blog!