Amazon Nova In Action - Canvas and Reel

Estimated Reading Time: 8 minutes
Last Updated: 2025-01-24

Disclaimer: The details described in this post are the results of my own research and investigations. While some effort has been expended in ensuring their accuracy - with ubiquitous references to source material - I cannot guarantee that accuracy. The views and opinions expressed on this blog are my own and do not necessarily reflect the views of any organization I am associated with, past or present.

In the previous post - make sure to read that one here first - we introduced a fictional company called Cardboard & Beyond which sells a single product: a cardboard box. We have been tasked to leverage Amazon Nova - a suite of state-of-the-art foundation models from AWS - to derive value from daily user feedback.

Our implementation plan is as follows:

We have already explored the “Understanding” models of Micro, Lite and Pro to perform feedback summarization and image/video analysis. Now we will turn our attention to generating media from that feedback.

Canvas

Nova Canvas is the first of the ‘Creative Content Generation’ models in the Nova suite^[1], offering Text-to-image generation (with/without image conditioning), Color-guided image generation, Image variation, Inpainting, Outpainting and Background removal^[2].

Exciting stuff!

In our case, amongst the choice words in the feedback from our fictional customers, we sometimes receive suggestions to improve our products at Cardboard & Beyond:

Wouldn’t it be good to visualize suggested improvements quickly? That’s where Nova Canvas can come in. By leveraging the Image Variation ability we can provide Canvas with an image and then ask it to generate a variation of it based on the suggestion.

Setup

Here is the call to Nova Canvas via the Bedrock Invoke API:

# Define the model ID for Nova Canvas

MODEL_ID = 'amazon.nova-canvas-v1:0'
images_list = []
images_list.append(image_body)

# Prepare the request payload
payload = {
            "imageVariationParams": {
                "images": images_list,
                "text": "Make the box green, photo-realistic",
                "negativeText": "low resolution, bad quality, without text",
                "similarityStrength": 0.7,
            },
            "taskType": "IMAGE_VARIATION",
            "imageGenerationConfig": {
                "cfgScale": 8.0,
                "width": 1280,
                "height": 720,
                "numberOfImages": 4
            }
        }
# Invoke the model
try:
    response = bedrock_runtime_client.invoke_model(
        modelId=MODEL_ID,
        body=json.dumps(payload),
        accept="application/json",
        contentType="application/json",
        trace="ENABLED"
    )
except Exception as e:
    print(str(e))
    raise

The payload options you need to fill in will depend heavily on the Task Type. The Task Type we are using is IMAGE_VARIATION, the idea being that we want something similar to our existing product without going too avant-garde.

We can provide a list of source images for Canvas to consider and a text suggestion to interpret. In our case, we only have one sample image:

Our text prompt is simple: make the box green and make it photo-realistic. Alongside the text prompt is the negativeText, in which we write what we don’t want. Finally, the similarityStrength parameter indicates how close the generated images should be to the source image(s)^[2].

The imageGenerationConfig identifies how many different images to create, the output dimensions, and other properties.

Results

Here are some of the generated images from Canvas for the above image and prompt:

Putting the Bob Ross vista to one side, Canvas created the concept designs we were looking for, with various types of green boxes generated for us. On a larger scale, this could be pretty useful for generating ideas based on user feedback. However, as is common with image-generation foundation models, text is not its forte. It is disappointing that part of the prompt made its way directly into the generated images, especially as the negativeText entry stated ‘without text’.

The careful reader may have seen that these results were generated with a similarity strength of 0.7. What would happen with a similarity strength of 1? Let’s find out:

A similarity strength of 1.0 has done exactly what it said it would. The source image was a simple cardboard box, and the images in general reflect the same color. If you compare the two result sets, you’ll notice that the images are very similar in viewpoint, box size and content. I believe this is due to the seed value for the payload in both, which was defaulted to 12. Different seeds will produce different images.

Let’s try a different seed value of 100 and turn the similarity strength down all the way to its minimum of 0.2. We should expect images quite different to our original source.

…I think we’ll move on.

Reel

We have used Amazon Nova to analyze and extend our data, but we can do even better with the final model: Amazon Nova Reel, a “state-of-the-art” text and image to video generator^[1].

Throughout all of my testing, Reel contributed the most expense. While Amazon Nova Canvas can cost up to $0.08 an image, Nova Reel costs $0.08 per second of video generated - quite steep. Therefore, caution is warranted when considering both of these services at scale.

Cardboard & Beyond like to know how their customers are feeling about their cardboard boxes every day, and want to display a video on a monitor in their offices which reflects the sentiment (positive, neutral, negative) of customer feedback. We have been asked to find a way to do this.

Earlier, when we used Amazon Micro to determine the sentiment of the feedback for a given day, we had results like this:

We can feed these daily sentiments into Amazon Nova Reel to generate a video for the day via this process:

This is an example of how you can chain foundation model outputs together within a workflow. In the future, I will explore this in much more detail.

Setup

The call to Bedrock for Nova Reel is very straightforward:

# Define the model ID for Nova Reel
MODEL_ID = 'amazon.nova-reel-v1:0'
# Prepare the request payload
payload = {
    "taskType": "TEXT_VIDEO",
    "textToVideoParams": {
        "text": f"Create video about a cardboard box maximally expressing the emotion of the following feedback: {sentiment_summary}. Include negative, neutral or positive imagery based on the feedback sentiment"
    },
    "videoGenerationConfig": {
        "durationSeconds": 6,
        "fps": 24,
        "dimension": "1280x720",
        "seed": 0
    }
}

# Invoke the model
try:
    invocation = bedrock_runtime_client.start_async_invoke(
        modelId=MODEL_ID,
        modelInput=payload,
        outputDataConfig={
            "s3OutputDataConfig": {
                "s3Uri": f"s3://productfeedback-media-us-east-1-<redacted>/generated_videos/year={feedback_date.year}/month={feedback_date.month}/day={feedback_date.day}/"
            }
        }
    )

In the payload, you simply provide a text description of the video you want to create. Here, I pass in the sentiment summary for the day, asking it to change the imagery based on whether the sentiment is positive, neutral or negative. Then, we set the properties of the video to be created, including its length. A 6-second video costs $0.48!

Where this call differs to previous Bedrock invocations is that it is asynchronous, meaning that you invoke the request to Bedrock to start creating the video, but program execution does not wait until the video creation is finished - you won’t receive bytes for your video in the API response. Instead, you provide an S3 bucket and prefix to which Bedrock can deliver the final videos.

Results

Let’s have a look at the generated videos.

Positive sentiment

Prompt summary: Multiple users expressed satisfaction with their purchase, highlighting the product’s sturdiness and functionality.

Neutral sentiment

Prompt summary: Most users have neutral to indifferent feelings about the product, describing it as just a cardboard box with nothing exciting.

Negative sentiment

Prompt summary: Multiple users expressed extreme dissatisfaction with the product’s quality and usefulness, describing it as the worst they’ve ever purchased.

Well, these are adorable. You could definitely picture these videos on loop in an office somewhere.

If you are wondering why they look very similar, this is because they were generated with the same seed value.

Conclusion

In this post, we have shown an example of how Amazon Nova’s Creative Content Generation models can be leveraged and chained in sequence to generate new concepts from customer data within a fictional company. Obviously, this scenario is a contrived one - it is unlikely that all Nova models will be used in a company workflow at the same time, but I hope I have shown that there are lots of creative ways you can use tools like Amazon Nova to your benefit.

Are the models the best for your use case? I have no idea, but it’s certainly worth trying them out!

References

[1] - Amazon Web Services. Amazon Nova Creative Content Generation Models. https://aws.amazon.com/ai/generative-ai/nova/creative/

[2] - Amazon Web Services. Request and response structure for image generation. https://docs.aws.amazon.com/nova/latest/userguide/image-gen-req-resp-structure.html