Solidity patterns to save gas on blockchain storage

In this article, I will discuss how the storage works on the EVM machine, and how can we write smart contracts that enable saving more on blockchain gas fees. The gas fees occur when we or any user interacts with the smart contracts that mutate the state of information on the blockchain, like transferring tokens to another person.

Storage in EVM

The topic of storage in Ethereum Virtual Machines (EVM) is a fundamental aspect of Ethereum smart contracts. Here’s an overview of how storage works in the EVM:

What is EVM?

The Ethereum Virtual Machine (EVM) is the runtime environment for Ethereum smart contracts. It’s a quasi-Turing complete machine; in other words, it can execute most computations given enough resources (like gas)

Storage Model

The Storage Model in the Ethereum Virtual Machine (EVM) is a key component to determine how data is stored and managed. This model differs significantly from traditional computing systems and plays a crucial role in executing smart contracts. Each Ethereum account, including user accounts and smart contracts, possesses its own storage space, which can be considered a virtual hard drive on the blockchain. The comprehension of this concept directly impacts the proper smart contract development by blockchain programmers. We can look at the EVM storage model as outlined below:

1. Account-Based Model: Unlike Bitcoin’s UTXO model, Ethereum uses an account-based model. Each account has a persistent state and storage.
2. Storage Space: Each account has associated storage, which is like a virtual hard drive. This storage is practically infinite, but writing to it and modifying it incurs gas costs.
3. Storage Slots: Storage is organized into slots. Each slot can hold 256 bits of data. This is a key consideration in smart contract development, especially for optimizing storage to minimize gas costs.

Optimization Strategies

The patterns we discuss today will enable us to be aware of `storage slots` and better optimise these slots for further gas optimisations. Given the gas costs associated with storage operations, developers must employ various techniques to optimize their contracts’ storage usage. Given the gas costs associated with storage operations, developers must employ various techniques to optimize their contracts’ storage usage. We minimize the gas costs associated with EVM storage by structuring data efficiently and understanding the nuances of how different operations consume gas. This approach benefits the contract creators in terms of deployment and maintenance costs that can be outlined below: 

1. Efficient Use of Storage: Due to high gas costs, it’s crucial to use storage efficiently. This includes packing variables into slots and minimizing the number of storage writes.
2. Memory vs. Storage: Memory is a temporary place to store data and is wiped clean between external function calls. It’s cheaper to use than storage, so temporary data should be stored in memory, not storage.
3. State Variables: Variables declared outside of functions are stored in the contract’s storage. Their layout in the contract can affect how many storage slots are used.

Before we start with introducing the pattern, let’s keep in mind that Solidity, as the primary language for writing Ethereum smart contracts, offers various features and nuances when it comes to handling storage. All solidity developers must know the implications of using diverse data structures like mappings and dynamic arrays, how EVM stores different data types, and understand the optimization that the Solidity compiler applies to storage usage. To write more efficient, cost-effective, and robust smart contracts, developers would extremely benefit from mastering these concepts.

1. Data Types and Storage: In Solidity (Ethereum’s primary language), different data types consume different amounts of storage space. Understanding these can help in optimizing for lower gas usage.
2. Variable Packing: Solidity tries to pack smaller data types into a single 256-bit storage slot to save space and gas.
3. Mappings and Dynamic Arrays: Mappings and dynamically-sized arrays in Solidity do not have a set limit on size, but adding elements to them increases gas costs.

Gas Saving Patterns

The following patterns can be enhanced and used outside the box

Storage Limiting

Using blockchain storage in Solidity is notably costly. To optimize for gas savings, it’s crucial to use memory for storing temporary data during function execution since memory is a less expensive resource compared to storage. Additionally, minimizing the frequency of storage updates can significantly reduce gas costs. For instance, instead of updating storage variables multiple times within a function, you can calculate the final result in memory and update the storage only once at the end. This approach reduces the number of state changes on the blockchain, which are expensive operations, leading to lower transaction costs.

contract LimitStorageExample {
    uint public result;

    function calculate(uint[] memory data) public {
        uint tempResult = 0;
        for (uint i = 0; i < data.length; i++) {
            tempResult += data[i];  // Perform calculations in memory
        }
        result = tempResult;  // Update storage only once
    }
}

Variable Packing

Ethereum’s storage is organized into 256-bit slots, and even smaller data types like uint8 occupy an entire slot. However, by placing similar small-sized data types consecutively in your contract, Solidity’s compiler can pack these into a single storage slot, optimizing space and saving gas.

Visual Explanation: Imagine a storage slot as a container that can hold up to 256 bits. An address type, for example, occupies 160 bits. If you declare two address variables consecutively in your contract, they can both fit into a single 256-bit slot, instead of each taking up a separate slot. This efficient packing of variables into fewer slots leads to a more gas-efficient contract.

In the bellow table, you can look up the amount of bits each variable type will occupy on a storage slot.

Note that the packing of smaller types is efficient only when they are declared consecutively. The Solidity compiler will automatically pack these variables into a single 256-bit slot where possible. otherwise, the rest of the storage slot will be padded until its 256-bit is filled with blank space. the goal for the developer must be to reduce the padding and increase the packing to fit the variables into the 256-bit slot completely.

Here’s an example of how NOT to do it:

contract NotGoodPackingAddressAndBooleanExample {
    address public userAddress;  // 160 bits
    bool public isActive;        // 8 bits
    uint96 public unused;        // 96 bits padding will make it fill 2 storage slots instead of only 1

    // Other variables follow to fill the padding required to be placed CONSECUTIVELY ...
}

In this contract, userAddress occupies the first 160 bits of a storage slot. isActive takes up 8 bits as that is the default amount for a boolean, but Solidity doesn’t allow tight packing of non-integer types with other variables directly. And since we used a uint96 , which is bigger than 256 when added to 160 + 8 bits, will result in the rest of the 1st slot being padded so the 96-bit can be added to the 2nd storage slot. Therefore, resulting in 2 storage slots being used with unused to fill up the rest of the slot spaces, ensuring no space is wasted within this 256-bit slot. As you can guess, this isn’t a very effective way of saving gas, is it?

So here are improved variations that can be referenced in the below examples:

pragma solidity ^0.8.0;

contract BetterVariablePackingExample {
    // First variable: address (160 bits)
    address public userAddress;

    // Second variable: uint88 (88 bits)
    uint88 public userBalance;

    // Third variable: bool (should take 1 bit but takes 8 by default)
    bool public flag1; 

    /// @notice Up to 8 boolean variables can be further optimised by "Boolean Packing" pattern
    bool public flag2; 
    bool public flag3;
    bool public flag4;
    bool public flag5;
    bool public flag6;
    bool public flag7;
    bool public flag8;

    // Additional state variables or functions
}

pragma solidity ^0.8.0;


contract BestVariablePackingExample {
    // Declare an address variable (160 bits)
    address public userAddress;

    // Immediately after, declare a uint96 variable (96 bits)
    // These two will be packed into the same 256-bit storage slot
    uint96 public balance;

    // Other state variables or functions
}

When dealing with complex data types in Solidity, understanding their storage implications is essential for efficient smart contract development. In the Ethereum Virtual Machine (EVM), the way data is stored can greatly impact the gas costs associated with a contract’s execution and deployment. Two such complex types are structs and arrays, both of which have variable sizes that depend on their contents. Structs are custom-defined types that can combine various other types, and their size is dictated by the aggregate of their components. Arrays, on the other hand, can be of fixed size or dynamic, with the latter adjusting in size as elements are added or removed. Moving on to dynamic types, bytes and string are special cases. These types are dynamically sized, meaning they can expand to accommodate data as needed. However, this flexibility comes with a unique storage model: both bytes and string allocate a separate storage slot to hold the length of the data, with the actual data stored in subsequent slots. Lastly, Solidity also includes fixed-point numerical types, known as fixed and ufixed. These types allow for decimal numbers but are not frequently used due to their complexity and the higher gas costs involved in operations. Understanding these distinctions in data types and their storage patterns is key to optimizing smart contract performance and cost.

• Structs and Arrays: The size of structs and arrays can vary greatly. A struct’s size depends on its constituent elements, while arrays can be either fixed-size or dynamic.
• Dynamic Sizes: bytes and string are dynamically sized and use a separate slot to store their length. The data itself is stored in subsequent slots.
• Fixed and Ufixed: These are fixed-point numerical types and are not commonly used in Solidity due to the complexity and gas costs associated with them.

Booleans Packing

In Solidity, boolean values are stored as uint8, taking up more space than necessary. i.e. The default storage size for a boolean (bool) variables is the same as uint8, which means each boolean consumes 8 bits of storage. However, a boolean value intrinsically requires only 1 bit (since it represents just true or false). This discrepancy in storage size can lead to inefficiencies, especially when dealing with multiple boolean variables.

Problem Overview:

• Default Storage Size: Each bool is stored as uint8, taking 8 bits of space.
• Inefficiency with Multiple Booleans: If you have, say, 32 boolean variables, and each is stored as uint8, you’re effectively using 256 bits (32 booleans * 8 bits each) instead of just 32 bits (1 bit per boolean), which is inefficient.

Solution: Packing Booleans

• Concept: The solution is to pack these boolean values into a single uint256 variable. A uint256 variable can store up to 256 bits, allowing us to efficiently pack up to 256 boolean values in it.
• Implementation: Create functions to handle the packing and unpacking of boolean values into and from this uint256 variable.
• Gas Efficiency: This approach is more gas-efficient, as it reduces the number of storage slots used. Operations on a single uint256 variable is generally cheaper than using separate storage for each boolean.

Visual Explanation: A uint256 can hold up to 256 bits, equivalent to 256 boolean values. Therefore, instead of using 256 separate uint8 slots (each for a boolean), all 256 boolean values can be packed into a single uint256 slot. This method not only saves a substantial amount of space but also results in lower gas costs due to fewer storage operations.

Here are a few examples that depict the implementation:

1. Packing and Unpacking Booleans into uint256 :

pragma solidity ^0.8.0;

contract BooleanPacking {
    uint256 private packedBooleans;

    function setBoolean(uint index, bool value) public {
        require(index < 256, "Index out of bounds");
        
        if (value) {
            // Set the bit at the position to 1
            packedBooleans |= (1 << index);
        } else {
            // Set the bit at the position to 0
            packedBooleans &= ~(1 << index);
        }
    }

    function getBoolean(uint index) public view returns (bool) {
        require(index < 256, "Index out of bounds");
        return (packedBooleans & (1 << index)) != 0;
    }
}

In the above example of BooleanPacking contract,

• setBoolean function sets the boolean value at the specified index.
• getBoolean function retrieves the boolean value at the specified index.
• The require statement ensures the index is within the range (0-255).
• The bitwise operations are used for setting and getting the specific bit.

2. Example with 32 Booleans or more:

contract ThirtyTwoBooleans {
    uint32 private packedBooleans;  // Only using 32 bits

    // Similar setBoolean and getBoolean functions can be implemented
}

3. In this one, we revisit the BetterVariablePackingExample contract in the Variable Packing examples and optimise it further using the Boolean Packing pattern:

pragma solidity ^0.8.0;

contract BestPackingExample {
    // First variable: address (160 bits)
    address public userAddress;

    // Second variable: uint88 (88 bits) for example this is a sufficant number for a user ballance
    /// @notice: for example this is a sufficant value type for a user ballance
    uint88 public userBalance;

    // Single uint8 to store up to 8 boolean values (1 bit each)
    uint8 private packedBooleans;

    // Function to set a boolean value at a specific position
    function setFlag(uint index, bool value) public {
        require(index < 8, "Index out of bounds");

        if (value) {
            // Set the bit at the position to 1
            packedBooleans |= (1 << index);
        } else {
            // Set the bit at the position to 0
            packedBooleans &= ~(1 << index);
        }
    }

    // Function to get a boolean value at a specific position
    function getFlag(uint index) public view returns (bool) {
        require(index < 8, "Index out of bounds");
        return (packedBooleans & (1 << index)) != 0;
    }

    // Additional state variables or functions
}

In this revised contract:

• The packedBooleans variable is a uint8 that efficiently stores up to 8 boolean values.
• setFlag function allows setting a boolean value at a given index (0 to 7).
• getFlag function retrieves the boolean value at a specified index.
• The require statements ensure that the index provided is within the valid range for our 8 packed booleans.

This optimization reduces the number of storage slots used for boolean variables, as multiple booleans are now packed into a single uint8 variable, following the principle of efficient storage usage in Solidity. Please note that it’s a best practice to provide documentation for each boolean and what each index refers to. So you and other developers can have a point of reference for each index of the packed boolean values.

Therefore, here’s the complete example with documented comments:

pragma solidity ^0.8.0;

/**
 * @title PackingExample
 * @dev Demonstrates efficient storage of multiple boolean values in Solidity.
 */
contract PackingExample {
    // Address of the user (160 bits)
    address public userAddress;

    // Balance of the user (88 bits)
    uint88 public userBalance;

    // A uint8 variable to pack up to 8 boolean flags (1 bit each)
    // Each bit in this byte represents a different flag.
    uint8 private packedBooleans;


    /**
     * @dev Sets a boolean flag at a specific index.
     * @param index The index of the flag to set (0 to 7).
     * @param value The boolean value to set at the specified index.
     * 
     * Index Mapping:
     * 0 -> flag1 (e.g., isUserActive)
     * 1 -> flag2 (e.g., hasUserVerifiedEmail)
     * 2 -> flag3 (e.g., isUserPremiumMember)
     * ... and so on for each flag
     */
    function setFlag(uint index, bool value) public {
        require(index < 8, "Index out of bounds");

        if (value) {
            // Set the bit at the position to 1
            packedBooleans |= (1 << index);
        } else {
            // Set the bit at the position to 0
            packedBooleans &= ~(1 << index);
        }
    }


    /**
     * @dev Gets the boolean value of a flag at a specific index.
     * @param index The index of the flag to get (0 to 7).
     * @return The boolean value of the flag at the specified index.
     */
    function getFlag(uint index) public view returns (bool) {
        require(index < 8, "Index out of bounds");
        return (packedBooleans & (1 << index)) != 0;
    }


    // Additional state variables or functions
}

The above example contract includes in-depth comments for the sample boolean packing implementation, complete with thorough documentation for each index of boolean flags.

Conclusion

In conclusion, understanding and effectively applying storage optimization techniques in Solidity is key to developing robust, efficient, and gas-cost-effective smart contracts. Whether you’re choosing the right data types, efficiently packing variables, or understanding the nuances of dynamic storage, each aspect plays a pivotal role in the overall performance and user experience of a smart contract on the Ethereum, Layer 2, and/or other EVM-based networks.

We encourage developers to continually explore and apply these optimization strategies, keep updated about the latest developments in Solidity and EVM, and contribute to more efficient and sustainable blockchain ecosystems.