blockchain implementation

CHAPTER 12 Integrating Non-Blockchain Apps with Ethereum 205

Chapter 12 Integrating Non- Blockchain Apps with Ethereum

Although you can build entirely blockchain-based applications, it is far more likely that your applications will be a combination of traditional and blockchain components. You learn in Chapter 3 that some use cases lend themselves well to blockchain apps but others do not. In this book, we chose to highlight one use case, supply chain, because blockchain offers clear advantages over traditional methods. However, even a comprehensive supply chain applica- tion will likely run partially as a traditional application and partially on the blockchain.

Many emerging blockchain apps consist of core components that operate as smart contracts and other components that operate as traditional applications that interact with users and provide supporting functionality. This hybrid approach to application development requires the capability to integrate the two different development models. In other words, to develop hybrid applications that run par- tially on the blockchain, you need to know how to design them to talk with each other and operate seamlessly.

IN THIS CHAPTER

» Exploring differences between blockchain and databases

» Identifying differences between blockchain and traditional applications

» Integrating traditional applications with Ethereum

» Testing and deploying integrated blockchain apps

206 PART 4 Testing and Deploying Ethereum Apps

Distributed application design and development isn’t new. In fact, some of the difficulties with distributed applications led to the need for technologies like blockchain. Remember that blockchain technology doesn’t solve all application problems, but it does have its place. Now that you know how to develop dApps for the Ethereum blockchain, in this chapter you learn how to integrate your smart contracts with applications that do not include blockchain technology. The capa- bility to integrate blockchain and non-blockchain applications makes it possible to develop applications that use the right technology for a wide range of needs.

Comparing Blockchain and Database Storage

In Chapter 2, you learn about some of the differences between storing data in a blockchain and a database. Both technologies can store data, but clear differences exist between the two. One of the first obstacles you might encounter when asked to integrate blockchain with an existing application is determining what data you should migrate to the blockchain.

Traditional applications store most of their data in a database. Databases provide fast access to shared data. Blockchains can also provide access to shared data, but they may not be as fast as a database. As you learn in Chapter 2, there are other differences as well. It is important that you understand the relative strengths of each data storage technique to make good design decisions for integrating block- chain into your organization.

When you begin the design process for integrating new blockchain apps with existing non-blockchain apps, determine the best home for each type of data based on how you plan to use it. For example, it doesn’t make sense to store on the blockchain low-importance data that you update regularly. However, if it is important that you maintain a historical record of all changes to that data, the blockchain might be a good place for it. Always remember that a cost is associated with writing to the blockchain.

The rest of this section lists the most important features that highlight the differ- ences between databases and blockchains. Understanding the effect of each dif- ference will help you to design hybrid integrated applications that meet your organization’s goals.

CHAPTER 12 Integrating Non-Blockchain Apps with Ethereum 207

Locating control Databases are central repositories of data that are shared by a collection of local and remote clients. The database administrator controls access to the database and manages changes to the database’s content and format. Although clients external to the database owner’s organization may be permitted to access and update the data, a central authority controls the database and its content.

Blockchains are ledgers of data shared among many nodes. There is no central copy of the blockchain data. All copies are the same. The blockchain technology guarantees that each node verifies the integrity of the blockchain data and can easily detect unauthorized changes. With public blockchains, any node, and its users, can access the blockchain data without requiring specific permission do to so. Permissioned, or private, blockchains impose access restrictions and a more traditional central access control model.

Imposing data format Databases (at least relational databases) are collections of tables that each contain data in similar formats. Each row of data in a relational database has the same data format. Likewise, each column of a table contains a list of items of the same data type. A database’s schema describes the format of the tables in that database. The database administrator maintains the database schema and con- trols any changes to it. Database applications can count on the fact that when they read data from the database, it conforms to the current database schema.

Blockchain does not enforce any data format for data stored in its blocks. Smart con- tracts may define formats for the data each one stores, but any block on the block- chain may contain transaction data created by many different smart contracts.

Updating data Traditional databases support the classic CRUD operations: Create, Read, Update, and Delete. Databases depend on the capability of each client, based on granted permissions, to be able to create and manage data in the database. Part of the data management process includes the capability to update and delete data. The cur- rent state of the database stores only the latest version of any table row’s data. The capability to overwrite data reduces redundancy and confusion over multiple data versions.

Blockchain technology supports only two operations: verifying a transaction and writing data to the blockchain. After you write data to the blockchain, it is immu- table. Blockchain technology does not support update and delete operations. The

208 PART 4 Testing and Deploying Ethereum Apps

only way you can update data is to add new data to the blockchain that supersedes the previous data version.

Optimizing performance Database vendors include new performance features with every new release. The goal is to provide the fastest access to data stored in the database. Database appli- cation developers routinely analyze their database queries to ensure that they are optimizing their code for the fastest data access. In most cases, slow queries are viewed as defects and become candidates for modification.

Blockchain technology is not generally focused on performance. In fact, block- chain has sometimes been referred to as a slow database. Although this compari- son is incomplete and unfair, it is generally accurate in that blockchain data access is slower than equivalent database access. The distributed nature of blockchain data, along with its integrity guarantees, mean that blockchain data storage will continue to be slower than database storage for the foreseeable future.

Protecting confidentiality Traditional databases benefit from their central control. The database administra- tor can restrict access to any data to only authorized individuals. Most database management systems provide built-in table and row-based permissions, and some include column-based permissions as well. And most current database management systems provide mechanisms to encrypt part or all of the data in a database. These features can make it easy to enforce confidentiality.

A public (permissionless) blockchain does not enforce access controls for its data. Any user who has access to a blockchain network node can view the data the blockchain stores. Some blockchain apps use encryption to enforce confidential- ity, but managing keys in a distributed, trustless environment is challenging. These challenges have minimized widespread adoption of data encryption for blockchain data. Private, or permissioned, blockchains can provide a general level of confidentiality. Users must have blockchain access granted to them to access the private blockchain. However, after a user gains access to the Ethereum block- chain, that user can access all blocks, just like in a public Ethereum blockchain.

Paying for storage Many people view traditional database storages as free and blockchain storage as costing cryptocurrency. This view is only partly correct. Although it is true that every database operation does not have a direct cost associated with it, the infra- structure on which the database operates is not free. Setting up and running a

CHAPTER 12 Integrating Non-Blockchain Apps with Ethereum 209

database server requires a substantial investment. Hardware, software licensing, software development, and personnel costs can be high.

Blockchain apps have software development costs but generally far lower infra- structure requirements. To compensate nodes for contributing to the infrastruc- ture and mining operations, each transaction costs a small amount of cryptocurrency. In the case of costs, both technologies require investment.

Providing integrity and transparency Traditional databases do not provide data integrity or transparency by default. Multiple users can modify data, and even overwrite each other’s changes. Further, most database management systems do not log all data changes by default. These problems are well-known and have resulted in extensions to database access lan- guages and database management system to support integrity and transparency. Database transactions can help define scope of work to maintain a steady database state. For granular integrity, database administrators can enforce strict rules on who can modify data, and concurrent control mechanisms such as locking can help avoid data write collisions. To provide more transparency, many database management systems provide the capability to log selected data modifications in separate tables, providing audit trails for later inspection.

The fundamental design of blockchain technology provides integrity and trans- parency by default. The consensus mechanism provides sanctioned integrity for every write to the blockchain. And because the blockchain is immutable, it auto- matically keeps every version of every data item written to any block.

Protecting resilience A central database is the core data repository of many organizations. Although a central database is convenient, it provides a Single Point of Failure (SPoF). Failure of the database means no user can get the data he or she need and the application stops working. For some organizations, that situation would be catastrophic. Many organizations invest heavily in hardware, software, and personnel to main- tain current separate copies of their data for disaster purposes. Creating an infor- mation ecosystem that is resilient to failures of its primary data repository is expensive.

Blockchain technology depends on the distribution of a ledger across many nodes. Because the network nodes trust each copy of the ledger, that data is accessible through any node on the network. If any node fails, all of the other nodes can con- tinue to operate and the data is still available. The design of blockchain technology provides resilience by default.

210 PART 4 Testing and Deploying Ethereum Apps

Table 12-1 summarizes the differences between storing data in a database and on a blockchain.

Contrasting Execution and Flow in Blockchain dApps and Traditional Applications

A big difference doesn’t exist between traditional distributed applications and blockchain dApps. With traditional applications and distributed dApps, the soft- ware sets up an initial state, waits for user input, and responds to that input. The main difference is in where the application code operates and what component handles validation.

Traditional applications mostly operate on a small number of computers. Although some programs operate entirely on a single computer, it is more common for functionality to be split up among at least two computers. In this architecture, some parts of the application, such as the code that interacts with the user, runs on the client computer, while other code, such as code that interacts with a data- base, runs on a server computer. This client-server architecture is an older but still common architecture for software applications.

TABLE 12-1 Database Storage versus Blockchain Storage Feature Traditional Database Blockchain

Locating control Centralized control; one central data- base copy

Decentralized control; complete copy of the blockchain on each node

Imposing data format Data schema defines data format No schema; each smart contract decides how to store its data

Updating data Create, Read, Update, Delete (CRUD) Read, Write

Optimizing performance

Optimized for short response time and high-throughput

Not optimized for performance

Protecting confidentiality

Centrally managed permissions No default confidentiality for pubic blockchains

Paying for storage Up-front infrastructure costs Per-transaction costs

Providing integrity and transparency

Dependent on DBMS and application Consensus and immutability provide integrity

Providing resilience Possible with substantial investment Complete copy of the blockchain stored on each node

CHAPTER 12 Integrating Non-Blockchain Apps with Ethereum 211

Over the last two decades, reliance on the Internet and its resources has grown fast — and that growth is constantly accelerating. Internet resources were once primarily endpoints or information sources, but now a growing number of resources are computational components. Today, you can write applications that mostly call functions that run on other servers. That changes the overall flow of today’s applications. Instead of just executing a series of steps on a client or server, tasks may run on many remote computers or devices.

Blockchain dApps are really extensions of the distributed application model, with one important difference: Smart contract code runs not on one node but on all nodes. And any transaction that you create for the blockchain has to wait until a miner selects it and successfully completes the requirements of the blockchain to write that block to the blockchain.

The response time for any blockchain can be far longer than writing data to any other storage location. Although this longer write cycle may not change the flow of your application, it can affect how users perceive and use the application. You have to be aware of how blockchain operates to anticipate slower writes to the blockchain.

Whenever possible, avoid making users wait for blockchain operations to com- plete before moving on. You might have to redesign your user interface to allow users to carry out some tasks, but inform them that other tasks take longer. Per- haps you can allow users to submit data for the blockchain, and then allow them to do other things within the application while you’re waiting for the blockchain return status. After your application receives a blockchain return status, it can alert the user and provide a way for the user to view the status and choose to move to another step in the application.

When integrating distributed application components, one of the most important factors is considering how new components will affect users. A large part of your design activities should involve developing a design that best meets the needs of your users.

Designing Goals for Incorporating Blockchain into an Existing Application

The first item to consider when planning to integrate multiple applications is to define your goals for the integration. You have to be able to clearly explain why you’re integrating the applications in the first place. If you can’t explain your reasons for starting an integration project, you’ll likely encounter problems.

212 PART 4 Testing and Deploying Ethereum Apps

Although blockchain technology is one of the most talked about innovations of the last decade, that fact isn’t a good enough reason to embrace it. You learn in Chapter  3 about different use cases for blockchain technology. They aren’t the only ones, but they are examples of where blockchain can fit well.

No mandatory goals exist for integrating blockchain technology into another application, but a few high-level objectives should be part of any integration project. The following list is a collection of goals you should resolve for every migration project, especially those that involve integrating blockchain compo- nents. You can use this list as an initial checklist when planning to integrate blockchain technology with your existing applications:

» Address application shortcomings. Integrating any non-blockchain applica- tion with blockchain technology should solve one or more ongoing problems with the existing application. If your traditional application doesn’t have any unresolved problems, integrating a new technology may have little value. New technology should always be a way to solve existing problems.

» Introduce previously unavailable features. Your application may be fine as it is but unable to provide new functionality that your users want. For example, integrating a blockchain supply chain application could allow your users to see where their products are along the supply chain and trace purchased products back to their origin. Providing functionality that was previous unavailable is a potential reason to integrate blockchain technology.

» Enhance the user experience. Integrating any new technology should enhance the overall user experience, not harm it. You could argue that the previous supply chain example fits this category too. Giving users more visibility into the supply chain path provides a more complete picture of product status and enhances the user experience. This goal is also a warning that blockchain integration should avoid creating obstacles for users that reduce the application’s usefulness.

» Reduce operational costs. One of the main features of blockchain technol- ogy is its capability to offer disintermediation, which is the reduction of the reliance on intermediaries to control and manage transfers of items of value from one party to another. Blockchain technology should be able to offer ways to eliminate at least some of the middlemen in business transactions. With fewer middlemen charging service fees, overall operational costs of blockchain applications should be lower. Your design specifications should include statements of how much operational cost the integrated blockchain app components should reduce.

» Enhance auditability and compliance. One of the most obvious advantages of blockchain technology is in the context of auditing. The process of auditing organizations for compliance to various standards or regulatory requirements

CHAPTER 12 Integrating Non-Blockchain Apps with Ethereum 213

includes examining audit trails that represent organizational activities. You learn in Chapter 1 that blockchains are immutable, and all data ever written to it is maintained in its original state. Therefore, the data you move to a blockchain will automatically create its own audit trail. You can easily trace all of the data that any account has changed.

The most important goal when designing a blockchain integration solution is to articulate clear reasons why blockchain is better than what you have today. What does a blockchain solution offer that is superior to what you use today? If you can’t clearly explain the specific reasons why blockchain is better, it probably isn’t any better. As cool as blockchain is, it must solve a problem before it has value in any organization.

After you decide what you want the blockchain integration to provide, the next step is to decide how the integrated applications will work together to provide the fea- tures you want. Designing your integration has two starting points: You have existing smart contracts that provide the functionality that you want, or you must develop new smart contracts from scratch. Each approach has its own challenges.

Using existing smart contracts It’s a good idea to design the integration as if you were designing your smart con- tracts from scratch. If you already have smart contracts that provide the function- ality you want, you will already have a map of the data you need to provide and the functionality you can access. Even if you have existing smart contracts in place (perhaps you purchased code from a software vendor or acquired open-source code), you may still have the ability to modify and extend your smart contract code.

If you can change any existing smart contract code, start with the existing smart contract functions and data items, and add data and functionality as needed. If you can’t modify the smart contract code you’ll use, you must modify your exist- ing application to conform to the smart contract requirements.

Developing your own smart contracts Another approach to developing an integrated blockchain solution is to start with the existing traditional application and no smart contract code. This requires more design effort but gives you the most flexibility. If you choose this approach, care- fully consider what you want to move to the blockchain. Remember that you have to pay for transactions in a blockchain environment, and those transactions will complete more slowly than in a traditional database application environment.

214 PART 4 Testing and Deploying Ethereum Apps

For example, if you plan to migrate your core supply chain functionality to an Ethereum public blockchain to provide transparency, define the minimum data and functionality you’ll need to accomplish that task. Don’t put more in the block- chain environment than you really need. If you aren’t careful in what you put on the blockchain, you could increase operational costs, slow down your users, and leak more information than you intended. Always start with simple, streamlined features that you deploy to the blockchain, and build from there.

Identifying Interface Data and Transaction Requirements

You start developing a blockchain interface by examining your existing environ- ment and the goals of your project. You learned in the previous section that the degree of flexibility in your design depends on whether you have smart contract code to start with, and whether you can modify it. Once you know your starting point and what abilities you have to modify code in each environment, you can design the data and functional requirements for your interface.

An interface definition basically answers three questions:

» What do you want the interface to do? » What data must you provide to the interface? » What data will the interface return to you?

You answer the first question by defining smart contract functions. If your first requirement is to create a new product, you’ll probably need a function named createProduct(). It’s a good idea to name your functions as verbs, because they carry out actions. In most cases, your functions will create items, get items (that is, fetch and return data), or change the state of some items. (Of course, changing the state of any data really means adding some new data to the blockchain.) You should define all of the actions your new integrated blockchain dApp must carry out and define a function for each one.

After you define each of your new functions, you can answer the second question by defining the input parameters each function needs to carry out its intended purpose. As you design your interface, validate that all required data is either already available in the traditional application, can be generated by the traditional application, or is available by some other means in the blockchain environment. Don’t add data into your design unless you can provide that data.

CHAPTER 12 Integrating Non-Blockchain Apps with Ethereum 215

Finally, you must specify what data each function returns to the traditional appli- cation. You must identify what data your traditional application needs to maintain integrity with the blockchain data. In this case, some redundancy is required, as in the case of unique identification tokens, and other redundancy may be desired. For example, you may decide to keep copies of supply chain registrations in your local database to support fast lookups for your application’s users. Remember that if you do store any redundant data, you have to takes steps to ensure that it stays current and correct. You should only store redundant blockchain data if you can identify specific value in doing so.

After you define all functions, input requirements, and output requirements, you will be ready to write or modify the code to create your integrated blockchain application solution.

Creating or Modifying Contracts to Provide Data Interface

The process of writing smart contract code should be a smooth one, as long as you invested in the design process. A well-designed application is far easier to write than one that lacks a detailed design specification. The output of the design phase should include detailed function definitions, along with input and output require- ments, and specifications of what each function does and the state and local data each one needs to operate.

You’ll find that coding smart contracts with a detailed specification is little more than translating requirements into another language, Solidity. The design process is more than a simple translation exercise, but it is important that the design phase addresses as many of the development requirements as possible. A good design document leaves few questions for developers to answer. Comprehensive design documents give developers the ability to focus on the most efficient and effective ways to implement the required functionality.

Testing Integrated dApps Proper testing of integrated applications is an extension of individual testing of each participant application. In other words, you still have to test each individual appli- cation first. You should completely test all functionality of your traditional applica- tion, and then completely test your blockchain dApp. The testing requirements for integrated dApps depend on properly working components as a foundation.

216 PART 4 Testing and Deploying Ethereum Apps

After you’ve completed the testing each application component, you can move to integration testing. Integration testing is much like full dApp testing. You have to develop tests that run in the context of your existing application. Your integration tests should generate calls to each function you defined as part of the blockchain interface. Your tests should create normal calls, invalid calls, and calls that pass boundary data. Invalid calls include the following:

» Wrong function name » Wrong number or format of input parameters » Out of range or bad input data » Input data containing boundary values » Wrong expected output parameters » Attempt to call a function that isn’t visible » Smart contract function properly completed with return codes » Set a timeout for a function call that is too short » Reverse a transaction

Your calling application and smart contract code should properly handle each of these situations without generating failures or crashes. Your tests should ensure that all errors are handled in a manner that does not interrupt the normal flow of the application.

Deploying Integrated dApps Finally, after you are through the testing phase and ready to move your new inte- grated application to production, it’s time to deploy to a live blockchain. The deployment steps for the blockchain dApp are the same as the ones you used when you deployed your supply chain dApp. Truffle makes the deployment process easy. The difference is in the synchronization between the application components.

Because your traditional application will call functions in the blockchain dApp, you can’t update your traditional application until the dApp is deployed.

Technically, you can deploy the traditional application before deploying the dApp, as long as your updated traditional application has some configuration control that disables any interaction with the dApp. After you deploy the dApp, you change the configuration to allow the traditional application to communicate with the new dApp.

CHAPTER 12 Integrating Non-Blockchain Apps with Ethereum 217

Here are the three simple steps to deploying all of the components of an inte- grated blockchain app:

1. Deploy your fully tested dApp. 2. Deploy your fully tested updated traditional application. 3. Run any maintenance utilities required to synchronize initial data.

After both sides are up and running, you can (and should) run real-time tests to ensure that everything works in the live environment. The deployment process should be the easiest step in the process. As long as you start with a solid design, integrating traditional applications with blockchain dApps is an effective way to take advantage of blockchain’s many benefits.