petroleum engineering

Contents lists available at ScienceDirect

Fuel

journal homepage: www.elsevier.com/locate/fuel

Full Length Article

Prediction of methane adsorption in shale: Classical models and machine learning based models Meng Menga,⁎, Ruizhi Zhongb, Zhili Weic a Earth and Environmental Sciences Division, Los Alamos National Laboratory, Los Alamos, NM, USA b School of Chemical Engineering, The University of Queensland, Brisbane, Australia c Department of Earth and Atmospheric Sciences, University of Houston, Houston, TX, USA

A R T I C L E I N F O

Keywords: Shale gas Adsorption model Classical model Machine learning XGBoost

A B S T R A C T

Shale gas contributes significantly to current global energy consumption, and an accurate estimation of geolo- gical gas-in-place (GIP) determines an optimal production plan. As the dominant form of storage, adsorbed gas in shale formation is of primary importance to be assessed. This paper summarizes adsorption models into tradi- tional pressure/density dependent isothermal models, pressure and temperature unified model, and machine learning based models. Using a comprehensive experimental dataset, these models are applied to simulate shale gas adsorption under in-situ conditions. Results show that the modified Dubinin-Radushkevich (DR) model provides the optimal performance in traditional isothermal models. Pressure and temperature unified models make a breakthrough in isothermal conditions and can extrapolate the predictions beyond test ranges of tem- perature. Well-trained machine learning models not only break the limit of the isothermal condition and types of shale formation, but can also provide reasonable extrapolations beyond test ranges of temperature, total organic carbon (TOC), and moisture. Four popular machine learning algorithms are used, which include artificial neural network (ANN), random forest (RF), support vector machine (SVM), and extreme gradient boosting (XGBoost). The XGBoost model is found to provide the best results for predicting shale gas adsorption, and it can be con- veniently updated for broader applications with more available data. Overall, this paper demonstrates the capability of machine learning for prediction of shale gas adsorption, and the well-trained model can potentially be built into a large numerical frame to optimize production curves of shale gas.

1. Introduction

Shale gas has become an important contributor to the current global energy structure. In the U.S., the shale gas revolution has shifted the U.S. from an oil importer to an oil exporter. Natural gas production grew by 10.0 billion cubic feet per day (Bcf/d) in 2018 and the total production amount of shale gas occupies more than 60% of U.S. dry natural gas production [1]. In China, massive investments have been put into the exploration of shale gas. Shale gas production has increased by 40% between 2017 and 2018, reaching 20.0 million cubic meters per day [2]. Despite the widespread importance, substantial uncertainties exist in assessing the quantity of recoverable shale gas [3]. The primary uncertainty point is the assessment of natural gas adsorption under in situ conditions, which influences the determination of the geological gas-in-place (GIP) quantity and production life of shale gas wells. Natural gas is stored in the porous space of shale in three phases: free gas, adsorbed gas, and dissolved gas [3–9]. The adsorbed gas accounts

for 20 ~ 85% of total shale gas-in-place [3,4]. Therefore, the prediction of adsorbed gas is important for gas production.

Unlike coal bed methane, which is usually buried under 1000 m depth, most shale formations are located in a much deeper depth, from 1000 m to 3000 m. The reservoir pressure can go up to 30 MPa and the reservoir temperature can be as high as 90 °C [5,7]. Extensive adsorp- tion experiments of natural gas in shale have been conducted under different pressures and temperatures [6,8–12]. Based on these experi- mental data, an accurate prediction model for methane adsorption in shale is significant for estimating the GIP and proposing an optimal production plan. Most researchers [13–19] used classical isothermal functions to simulate shale gas adsorption. These classical models can mainly be divided into monolayer adsorption models (Langmuir style model), multilayer adsorption models (Toth model), and pore-filling models (DR model). Despite the wide application of traditional iso- thermal models, no one can deny their deficiencies when it comes to isothermal and homogenous assumptions. To overcome these

https://doi.org/10.1016/j.fuel.2020.118358 Received 7 March 2020; Received in revised form 3 June 2020; Accepted 8 June 2020

⁎ Corresponding author at: Los Alamos National Laboratory, EES-14 Geomechanics, USA. E-mail address: [email protected] (M. Meng).

Fuel 278 (2020) 118358

Available online 20 June 2020 0016-2361/ © 2020 Elsevier Ltd. All rights reserved.

T

deficiencies, Tang [3] innovatively used the idea of dual adsorption sites and proposed a dual-site Langmuir equation, which is similar to the dual-porosity model in poroelasticity theory of deformation-diffu- sion response of rock [20–22]. After comparing Langmuir-style models, Toth models, and DR style models, they concluded that the dual-site Langmuir model is superior to other models in interpreting observed phenomena and extrapolating adsorption isotherms beyond the test data [5]. In reality, shale has an abundance of different adsorption sites, and a two-site model still simplified the surface heterogeneous effect. Li [7] proposed a multi-site model and proved that their model is not only effective in predicting adsorption isotherms but also in the thermo- dynamic parameters of energetically heterogeneous shales.

The dual-site and multi-site models successfully break the limit of isothermal conditions. However, these models are mathematically more complex and require global optimization of several parameters. In ad- dition, they still cannot break the limit of shale types, and the relevant underlying assumptions should not be taken for granted [5]. These limits restrict the generality of these models in a large numerical frame of predicting and optimizing the production curves of shale gas. After years of development, machine learning has become a powerful tool to build predictive models [23–29] which can uncover hidden patterns and unknown correlations between variables. Currently, only a few

applications of machine learning have been applied to shale gas ad- sorption evaluation. Meng [28] attempted to use an ANN for predicting supercritical CO2 adsorption in coal. Results show that it not only provides similarly accurate results as Langmuir-style model, but also breaks the limit of isothermal condition and types of coal. Li [29] used a K-Nearest Neighbor algorithm to estimate the adsorbed shale gas con- tent by geological parameters. However, Li’s method ignored the la- boratory experimental data, which cannot be a complete substitution for adsorption experiments and can only be used for the rough esti- mation of gas content in the whole reservoir. Therefore, machine learning is a promising tool to predict shale gas adsorption but requires more work.

In this paper, we used a large amount of laboratory adsorption data to estimate geological GIP and production life of shale gas wells. A total of 630 adsorption experimental data were compiled and analyzed, with 278 data for classical mechanistic models and 352 data for machine learning models. Adsorption data may have different units, and they are unified to the same units for optimization [28]. The comparison be- tween classical models and machine learning based models were com- prehensively provided. Based on the results of optimized machine learning based models, this paper can predict shale adsorption behavior and be incorporated into the model of shale gas production plans.

Nonmenclaure

RMSE Root mean squared error SSE Sum of errors TOC Total organic content XGBoost Extreme gradient boosting ANN Artificial neural network RF Random forest SVM Support vector machine R2 Coefficient of determination ne Excessive adsorbed amount, mmol/g nabsolute Absolute adsorbed amount, mmol/g nmax Maximum adsorbed amount, mmol/g K 0 Temperature dependent Langmuir constant

Va The volume of the condensed phase per unit mass shale g The density of gas at a certain temperature and pressure a The density of gas at adsorbed phase freegas The density of free gas

k The parameter related to the affinity of coal for gas t Equilibrium constant which reflects the heterogeneity of

the adsorbent D A constant related to the affinity of coal for gas P Pressure, MPa K1,K2 Langmuir constant for dual-site Langmuir model A1,A2 The pre-exponential coefficient E1, E2 Constant of energy of adsorption R Ideal gas content

Table 1 Classical adsorption isotherm models.

Type No. Models

Langmuir style models Model 1 [18,19] =

+ n 1e

g a

nmax K P K P

0 1 0

(1)

Model 2 [14] = +

+ n kP1e

g a

nmax K P K P

0 1 0

(2)

Model 3 [5] = +

+ n kP1 1e

g a

nmax K P K P

g a

0 1 0

(3)

Model 4 [5] = +

n Ve nmax K P

K P a g 0

1 0 (4)

Toth models Model 5 [5] = +

n Ve nmax K P

K P t t a g

0 [1 ( 0 ) ]1

(5)

DR style models Model 6 [13]

=n n e1e max g a

D ln g a

2 (6)

Model 7 [14]

= +n n e kP1e max g a

D ln g a

2 (7)

Model 8 [14]

= +n n e kP1 1e max g a

D ln g a g

a

2 (8)

Notes: In the above equations, g is the density of gas at a certain temperature and pressure; a is the density of gas at adsorbed phase, which is usally assumed as a constant value; nmax is the maximum adsorbed amount; K 0 is the Langmuir constant; P is the pressure; k is the parameter related to the affinity of shale for gas; Va is the volume of the condensed phase per unit mass shale; t is equilibrium constant which reflects the heterogeneity of the adsorbent; D is a constant related to the affinity of shale for gas.

M. Meng, et al. Fuel 278 (2020) 118358

2

2. Methodology

2.1. Classical pressure/density dependent isothermal models

The most commonly used models for adsorption are pressure/den- sity dependent isothermal models, which use the pressure or density as the variable in the isothermal condition. Similar to the work by Tang [5], the most typical models are summarized in Table 1. All of them are optimized using the published adsorption data [5–8,30,31].

2.2. Pressure and temperature unified model

Different from the pressure/density dependent isothermal models, the pressure and temperature unified model breaks the limit of iso- thermal conditions. Tang [3] proposed a so-called “dual-site Langmuir model” including the influence of temperature, which is shown in Eq. (9).

= +

= =

+ +

( ) ( ) n n V

K T A K T A

(1 ) ( )

( ) exp , ( ) exp

e K P

K P K P

K P max max g

E RT

E RT

1 1

1 1 2 2

1 1

2 2

1 2 (9)

The temperature is introduced in this model by modification of the Langmuir constant K . Instead of considering K as a constant, Tang [3] used the temperature dependent equation to calculate K , which is

( )A exp ERT . The parameter is a weighted coefficient, which represents the fraction of the second type of adsorption site [3]. With this mod- ification, the temperature is included in the model and the new model is called the pressure and temperature unified model. Theoretically, this unified model should perform better with the verification of the ex- perimental data. The same set of data will be used for optimization as those applied to traditional isothermal models in Part 2.1.

2.3. Machine learning based models

In the field, there are various factors (variables) influencing the adsorption amount of methane in shale. These factors not only include external factors such as pressure, temperature, and moisture, but also internal factors such as total organic content (TOC), thermal maturity, and clay minerals. However, neither traditional isothermal models nor the pressure and temperature unified model can break the limitation of shale types (e.g., TOC, thermal maturity, clay components, moisture). With enough data, machine learning based models can include all the related variables. In the machine learning model, the prediction of methane adsorption on shale is a multivariate regression problem.

Although many variables contribute to methane adsorption on shale, it is not necessary to apply all these variables in the machine learning optimization. First, it is technically feasible—but physically kind of difficult—to obtain all the data. The measurement of moisture, TOC, and thermal maturity requires different equipment, in which in- dividual researchers usually do not perform all these basic measure- ments. That’s why these fundamental properties are not provided

completely in publications. Second, some variables, especially mineral components (TOC, thermal maturity, clay minerals), are not completely independent of each other. For example, Zhao [32] found a linear re- lationship between TOC and clay minerals for shale. Last but not least, the contributions of these variables to methane adsorption on shale are not equal. The external factors, such as pressure and temperature, are well-known and important. For other factors, Tan [4] mentioned that TOC is the primary controlling factor on methane sorption capacity, and clay minerals are secondary in importance on the methane sorption capacity compared to the TOC content.

Beaton [10–12] worked extensively on adsorption measurements of methane on shale. They provided the value of TOC for all different

Table 2 Optimized parameters and performance of isothermal models at 313.75 K using Chen’s data [8]

Types Parameters Performance

nmax K0 k Va t D SSE R2 AdjustedR2 RMSE

Model 1 5.3520 0.1786 0.1601 0.9805 0.9785 0.1265 Model 2 4.7910 0.2216 0.0058 0.1153 0.9859 0.9828 0.1132 Model 3 4.3630 0.2536 0.0215 0.1101 0.9866 0.9836 0.1106 Model 4 5.0650 0.2276 10.3200 0.1201 0.9853 0.9821 0.1155 Model 5 7.9290 0.2084 15.4000 0.6643 0.0965 0.9882 0.9838 0.1098 Model 6 4.7490 0.0965 0.1877 0.9771 0.9748 0.1370 Model 7 4.3150 0.0073 0.0855 0.0771 0.9906 0.9885 0.0925 Model 8 4.0460 0.0226 0.0813 0.0727 0.9911 0.9892 0.0899

Table 3 Best three isothermal models in all 20 cases.

Case ID Temperature Best three isothermal models

1 313.75 K [8] Model 8, 7, 5 2 333.75 K [8] Model 7, 2, 8 3 348.75 K [8] Model 7, 2, 8 4 368.75 K [8] Model 7, 8, 2 5 303.15 K [6] Model 7, 2, 8 6 323.15 K [6] Model 3, 7, 2 7 343.15 K [6] Model 2, 3, 8 8 303.15 K [7] Model 8, 7, 5 9 333.15 K [7] Model 5, 8, 7 10 363.75 K [7] Model 8, 5, 7 11 333.15 K [31] Model 3, 5, 4 12 373.15 K [31] Model 5, 3, 8 13 413.15 K [31] Model 5, 3, 8 14 318.15 K [5] Model 5, 8, 7 15 333.15 K [5] Model 5, 8, 7 16 348.15 K [5] Model 5, 8, 7 17 355.15 K [5] Model 5, 8, 7 18 313.75 K [30] Model 8, 7, 3 19 348.75 K [30] Model 5, 7, 8 20 368.75 K [30] Model 5, 8, 7

0

4

8

12

16

20

N u

m b

er o

f ca

se s

Types of model

Fig. 1. Number of best performance cases for different models.

M. Meng, et al. Fuel 278 (2020) 118358

3

tested shale, and the value of TOC is used to represent the internal factors. All major external factors of pressure, temperature, and moisture were also provided by Beaton [10–12]. Therefore, a total of 352 data points from Beaton [10–12] with four variables including pressure, temperature, moisture, and TOC are used as the input (Supplement 1). In this study, four types of popular machine learning

algorithms are used, including extreme gradient boosting (XGBoost), artificial neural network (ANN), random forest (RF), and support vector machine (SVM). The brief introductions of these algorithms are written as follows. The basic equations for these algorithms are suggested to be reached from the cited publications [28,33–36].

]6[atads’gnaiJ)b(]8[atads’nehC)a(

]13[atads’iL)d(]7[atads’iL)c(

]03[atads’gnoiX)f(]5[atads’gnaT)e(

0

0.5

1

1.5

2

2.5

3

0 20 40 60

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

313.75K Mod 313.75K Exp

333.75K Mod 333.75K Exp 348.75K Mod 348.75K Exp 368.75K Mod 368.75K Exp

0

0.02

0.04

0.06

0.08

0.1

0 10 20 30

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

303.15K Mod 303.15K Exp 323.15K Mod 323.15K Exp 343.15K Mod 343.15K Exp

0

0.01

0.02

0.03

0.04

0 5 10 15 20

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

303.15K Mod 303.15K Exp 333.15K Mod 333.15K Exp 363.15K Mod 363.15K Exp

0

0.02

0.04

0.06

0.08

0.1

0 20 40 60 80 A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

333.15K Mod 333.15K Exp 373.15K Mod 373.15K Exp 413.15K Mod 413.15K Exp

0

0.4

0.8

1.2

1.6

2

0 5 10 15

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

318.15K Mod 318.15K Exp 333.15K Mod 333.15K Exp 348.15K Mod 348.15K Exp 355.15 Mod 355.15K Exp

0

0.4

0.8

1.2

1.6

0 20 40 60

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

313.75 K Exp 313.75K Mod 348.75K Exp 348.75K Mod 368.75K Exp 368.75K Mod

Fig. 2. Performance of Model 8 for different groups of data. (Mod is model results; Exp is experimental results).

Table 4 Parameters and performance of Model 9 for different groups of data.

Parameters Chen's data [8] Jiang's data [6] Li's data [7] Li's data [31] Tang's data [5] Xiong's data [30]

α 0.42 0.999 0.368 0.802 0.806 0.678 A1 0.002 0.0024 0.002 5.71E-10 1.64E-04 0.029 A2 2.82E-07 1.92E-06 0.013 1.16E-02 2.78E-04 4.58E-07 E1 14.24 12.966 13.321 54.335 24.370 9.97E-14 E2 32.110 9.665 14.674 9.054 15.303 25.582 nmax 5.373 60.247 0.051 0.099 2.818 12.307 Vmax 12.075 282.043 0.163 0.269 8.878 42.708 SSE 1.076E-05 9.097E-05 1.076E-05 1.65E-04 0.002 1.258 R-square 0.9924 0.9952 0.9924 0.9932 0.9998 0.803 Adjusted R-square 0.9917 0.9944 0.9917 0.9864 0.9997 0.803 RMSE 0.0005 0.0017 0.0005 0.0019 0.0060 0.1571

M. Meng, et al. Fuel 278 (2020) 118358

4

2.3.1. Extreme gradient boosting (XGBoost) Developed by Chen and Guestrin [33], XGBoost is an open-source,

scalable end-to-end tree boosting system that is widely used by data scientists to achieve state-of-the-art accuracy on many classification and regression problems. Compared with other implementations of gradient boosting, XGBoost has proven to provide a faster and more accurate prediction. Gradient boosting is an approach where new models are created that predict the residuals of the prior model and then added together to make the final decision. It uses a gradient des- cent algorithm to minimize the loss when adding new models. XGBoost is a library for developing fast, high-performance gradient boosting tree models [34].

2.3.2. Artificial neural network (ANN) ANN is inspired by the biological neural networks of animal brains.

It is a powerful machine learning algorithm for both regression and classification problems. The deep neural networks have been applied to many fields including speech recognition, machine translation,

autonomous driving, etc. A typical ANN model consists of three parts: one input layer, one or several hidden layers, and one output layer [28]. In each layer, there are multiple nodes that receive values from the predecessor nodes, perform computations using activation functions, and then deliver the outputs to the successor nodes.

2.3.3. Random forest (RF) RF or random decision forests are an ensemble learning method for

classification, regression, and other tasks. It operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regres- sion) of the individual trees. Essentially, it aims to make the trees de- correlated and prune the trees by setting a stopping criterion for node splits [35]. RF has been successfully applied in many data science competitions and has the advantage of simplicity and interpretability.

2.3.4. Support vector Machine (SVM) SVM is a supervised learning model with associated learning

]6[atads’gnaiJ)b(]8[atads’nehC)a(

]13[atads’iL)d(]7[atads’iL)c(

]03[atads’gnoiX)f(]5[atads’gnaT)e(

0.00

0.50

1.00

1.50

2.00

2.50

3.00

0 20 40 60

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

313.75 K Mod 313.75K Exp 333.15K Mod 333.15K Exp 348.75K Mod 345.15K Exp 368.75K Mod 368.75K Exp 388.75K Mod 408.75K Mod 428.75K Mod

0

0.03

0.06

0.09

0 10 20 30

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

303.15K Mod 303.15K Exp 323.15K Mod 323.15K Exp 343.15K Mod 343.15K Exp 363.15K Mod 383.15K Mod 403.15K Mod

0

0.01

0.02

0.03

0.04

0 5 10 15 20

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

303.15K Mod 303.15K Exp 333.15K Mod 333.15K Exp 363.15K Mod 363.15K Exp 373.15K Mod 383.15K Mod 393.15K Mod 0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0 20 40 60

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

333.15K Mod 333.15K Exp 373.15K Mod 373.15K Exp 413.15K Mod 413.15K Exp 453.15K Mod

0

0.5

1

1.5

2

0 5 10 15

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

318.15K Mod 318.15K Exp 333.15K Mod 333.15K Exp 345.15K Mod 345.15K Exp 355.15K Mod 355.15K Exp 365.15K Mod 375.15K Mod 385.15K Mod

0

0.4

0.8

1.2

1.6

2

2.4

0 20 40 60

A d so rp ti o n

( m

m o

l/ g)

Pressure (MPa)

313.75 K Exp 313.75K Mod 348.75K Exp 348.75K Mod 368.75K Exp 368.75K Mod 388.75 K Mod

Fig. 3. Performance of the unified model for different groups of data. In the legend, Mod means model results and Exp means experimental results. Grey dashed lines are generated beyond the test range using the unified model.

M. Meng, et al. Fuel 278 (2020) 118358

5

algorithms for classification and regression analysis. When SVM is ap- plied in the regression, it is also called support vector regression (SVR). SVR uses the same principles as the SVM for classification, with only a few minor differences. In the case of regression, the output is a real number and a margin of tolerance is set in approximation. The main purpose of SVM is the same as SVR: to minimize error, individualizing the hyperplane which maximizes the margin [36].

2.4. Evaluation metrics systems

There are many forms of evaluation metrics systems, such as root mean squared error (RMSE), and coefficient of determination (R2). All these metrics offer complementary information that evaluates the per- formance of the optimization model. In this paper, four metrics systems are used.

RMSE has the following expression

= =n

y yRMSE 1 ( ) i

n

i i 1

2

(10)

The Sum of errors (SSE) is the sum of the differences between the prediction of each observation and the population mean.

= =

y ySSE ( ) i

n

i i 1 (11)

Coefficient of determination (R2 or R-square) has the following ex- pression

=R y y y y

1 ( ) ( )

i i

i i

2 2

2 (12)

3. Results and analysis

3.1. Results of traditional pressure/density dependent isothermal models

For isothermal models, the related parameters are different for various types of shale and various temperatures. To have a compre- hensive evaluation and comparison of the isothermal models, 20 groups of isothermal data from several different studies [5–8,30,31] are used (Supplement 2). Table 2 is an example of optimized parameters and performance of isothermal models at 313.73 K using Chen’s data [8]. In this case, all types of isothermal models provide satisfactory perfor- mance. But it does not mean that these models will always provide such good performance in all the 20 optimized cases. Detailed results of 20 cases are in Supplement 3. To have a better overview of the results, the

best three isothermal models were provided in all cases (Table 3). For different cases, the results of the best three models are different and they are presented in Fig. 1. According to Fig. 1, Model 8 provides the best performance in most cases compared with other models. Therefore, for the cases studied here, the modified DR models provide the best performance within these eight isothermal models. The performance of Model 8 for these cases is plotted in Fig. 2, and it shows that the pre- dictions are very close to the calculations.

3.2. Results of pressure and temperature unified model

For pressure and temperature unified model, the isothermal condi- tion is not required. Therefore, the same data used for classical models [5–8,30,31] are used for optimization and we have six groups of data. Using a global fitting method [5], the parameters and performance for these six groups of data are presented in Table 4. According to the evaluation metrics, the unified model shows excellent performances for the first five groups of data with the R-squares as high as 0.99. This is an excellent performance for engineering applications. The only exception is the last group of data [30], which only has an R-square of 0.803. The relatively poor performance may be caused by the errors in the ex- perimental procedure or the special characteristics of the shale samples performed in their tests.

Fig. 3 shows the performance of the unified model for all six groups of data. In the first five cases, the prediction curves are very close to the

Fig. 4. Machine learning optimization workflow.

Fig. 5. Division of training set and testing set in the machine learning optimi- zation.

Table 5 Hyperparameters tuning for different machine learning algorithms.

Model Tuned hyperparameters Range Best hyperparameters

Kernel linear, rbf, poly rbf SVM Regularization term C 10-1, 1, 10, 102 10

Kernel coefficient gamma

10-1, 1, 5, 10 1

RF Number of trees 5, 10, 30, 50, 100, 200 30 Max depth 5, 10, 15, 20, 30 20 Min sample split 2, 5, 10, 15, 20 2 Number of boosted trees 5, 10, 50, 100, 200 10 Max depth 3, 4, 5, 7, 9 4

XGBoost Learning rate 10-3, 10-2, 10-1, 1 10-2

Min child weight 1, 3, 5, 7, 9 3 Optimizer Adam, SGD, Adagrad Adagrad

ANN Kernel initializer Uniform, Glorot_uniform ffGloGlorot_uniform,

Uniform

Activation Relu, linear, softmax, tanh Relu Learning rate 0.0001, 0.001, 0.01, 0.1 0.01 Gradient descent 0.000001, 0.00001,

0.0001, 0.001, 0.01 0.00001

M. Meng, et al. Fuel 278 (2020) 118358

6

experimental data (points with different colors). Because both tem- perature and pressure are included in the model, this model has been successfully proven to make a breakthrough in the isothermal condi- tions. Another advantage of this unified model is it can extrapolate the predictions beyond test ranges. These extrapolations are shown as grey dashed curves in Fig. 3. For instance, in Fig. 3(a), reasonable predic- tions can be generated under temperatures of 388.75 K, 408.75 K, and 428.75 K using the optimized parameters. This feature cannot be rea- lized by any isothermal models. Hence, as mentioned by Tang [5], the unified model is not only superior to other available models in terms of interpreting the observed phenomena, but also can extrapolate ad- sorption isotherms beyond the test data.

3.3. Machine learning results

As mentioned before, Beaton’s data [10–12] are used for machine learning optimization. All major external factors such as pressure, temperature, and moisture were also provided by Beaton [10–12]. Therefore, in our machine learning optimization, four variables in- cluding pressure, temperature, moisture, and TOC are used as input. There is no ‘free lunch’ in machine learning or other data science op- timization, and each algorithm will be suitable for a different data set. In this paper, four machine learning algorithms are used for optimiza- tion: XGBoost, ANN, RF, and SVM. To evaluate the performance of these algorithms, a standard machine learning optimization workflow (Fig. 4) is implemented: (1) find enough data points and preprocess the data, including unit conversion, outliers detection, etc; (2) find enough data and divide the data set into training and testing data set. Usually, the training data set is further divided into a training set and a vali- dation set; (3) take into account that some parameters cannot be learned by machine learning algorithms and should be set before the

learning process begins. These parameters are called hyperparameters and can affect model performance [28,33–36], so it is necessary to perform cross-validation and grid search of hyperparameters using training data set; (4) use the optimized hyperparameters to train the model and generate a learning curve; (5) use the trained model to evaluate the testing data; (6) compare the performance of different algorithms using proper evaluation metrics, and choose the optimal algorithm; (7) check the robustness of the optimal model by re-dividing the training data and testing data, to see if we get the same optimal algorithm.

In the first step, we can plot all the data and delete the outliers. In our work, the data are from high precision indoor adsorption mea- surement, so no outliers are found. In the second step, we usually assign 80% of the data set as the training data and the remaining 20% data set as the testing data, which is a common rule in machine learning. Note that the data are randomly assigned and we should guarantee both the training set and the testing set cover the whole range of data set [37]. Fig. 5 shows the division of the training set and testing set from the whole data set. Both the testing set and training set lie in the whole range of the data set, which avoids the data bias. The cross-validation has been performed for each of these four algorithms and the best hy- perparameters are presented in Table 5. The range of hyperparameters is based on common numbers used by previous researchers [28,33–36].

Using the best hyperparameters, the learning curve can be plotted to show the performance of the training process. The performance mea- sure can be the loss function that is being optimized to train the model (such as logarithmic loss), or an external metric of interest to the pro- blem (such as RMSE). For different algorithms, we may use different metrics in the learning curves, and performance may be measured in different styles. For example, ANN is measured by optimization steps (epoch), whereas random forest is measured by the number of samples.

tserofmodnaR)2(krowtenlaruenlaicifitrA)1(

(3) Support vector m tsooBGX)4(enihca

Fig. 6. Learning curve of different machine learning algorithms using Beaton’s [10-12] data.

M. Meng, et al. Fuel 278 (2020) 118358

7

A bias-variance tradeoff is desired. As shown in Fig. 6, both ANN and XGBoost provide good performance because the training score and validation score are close to each other as the learning ends. For the SVM and RF, the validation curve is still decreasing. If more training

data is available, the model performs better. Fig. 7 shows the optimization results of these four algorithms using

Beaton’s data. In each plot, the x-axis is the adsorption from the ex- periments, and the y-axis is the predicted adsorption. The black dotted

gnitsetNNA)2(gniniartNNA)1(

gnitsetFR)4(gniniartFR)3(

gnitsetMVS)6(gniniartMVS)5(

(7) XGBoost training (8) XGBoost testing

Fig. 7. Training and testing performance of different machine learning algorithms using Beaton’s [10–12] data.

M. Meng, et al. Fuel 278 (2020) 118358

8

line is the 100% agreement line, and the points are optimized results. The closer the points are to the black dashed line, the more accurate the prediction of the machine learning algorithm. To quantitatively com- pare the performance of these machine learning algorithms, results of evaluation metrics are presented in Table 6. For the training set, all four algorithms provide good predictions (R2 > 0.94) and XGBoost pro- vides the best performance (R 12 ). For the testing set, XGBoost also provides the best performance with R2 of 0.99, which is much better than the other three algorithms. The performance of the trained XGBoost model is shown in Fig. 8 and the predictions are highly similar to experiments. To further check the robustness of the algorithm, we re- divide the training data and testing data for another four times. Then we provide the average values of performance of different algorithms, shown in Table 7. Still, we can see that XGBoost is the optimal algo- rithm.

As mentioned by Tang [5], a good model should have the prediction capability beyond the test data. To further test the XGBoost model, the predictions are extrapolated beyond test ranges. Fig. 9 (a) and Fig. 9(b) show the extrapolation of the temperature. The results are reasonable because, with higher temperatures, the adsorption content is lower [28]. Fig. 9(c) and (d) show the extrapolation of TOC, and our pre- dictions are consistent with early researchers [4]. The higher TOC, the larger adsorption amounts of shale gas. Fig. 9(e) and (f) show the ex- trapolation of moisture and, as expected, the adsorption ability de- creases as increasing of moisture [4,10–12], which is consistent with our knowledge. From all these cases, it is shown that the XGBoost model can extrapolate the predictions beyond test ranges with rea- sonable results. It is better to have experimental data to verify the

extrapolation capability. However, the experimental data used for machine learning is from Beaton’s study [10–12], which is unavailable for extrapolation. Furthermore, it is also not practical to use data from other studies to verify our experimental data. One reason is not all re- searchers provide complete experimental and sample conditions (e.g., pressure, temperature, moisture, TOC). Another reason is the shale properties from other studies may be different from the shale properties in Beaton’s study, which can significantly affect the methane absorp- tion. To possibly use other data for validation, it is suggested to include their shale data as an input. However, this requires further investiga- tion. Therefore, the experimental data to verify the extrapolation is unavailable in this study. Nevertheless, the trend is found to be rea- sonable based on previous studies [4,10–12,28]. Note that if the ex- perimental scale is large enough, it is possible to avoid the extrapola- tion.

4. Discussion and implications

Through the above analysis, the classical isothermal models have strong limitations in predicting adsorption of shale gas. Although they can provide relatively accurate results for a certain temperature, they are highly dependent on experimental results and are restricted by both temperature and types of shale. The pressure and temperature unified model can break the limit of the isothermal condition, and it can ex- trapolate the predictions beyond the test ranges of temperature. However, it is still limited to one specific type of shale. There are nu- merous types of shales in the world, and it will be perfect to find a model that can be used to predict the adsorption behavior of any given type of shale. But this kind of unique model is kind of unrealistic or unwise because the properties of different blocks of shale can be sig- nificantly different, and there are many properties that can influence the adsorption results. Using machine learning methods, a prediction model is still realistic for certain blocks of shale if a sufficient amount of data is available. In this study, using the comprehensive data from Beaton [10–12], the XGBoost provides the best performance, and it can successfully break the limit of temperature and shale type. Also, it provides accurate extrapolations beyond test ranges of both tempera- ture and shale type. In other words, using the trained machine learning model, theoretically, researchers can predict the adsorption behavior for a certain type of shale. Furthermore, the machine learning model can be conveniently updated. If a new dataset is provided, the current model can be upgraded with the new data set and provide broader applications. Therefore, compared with previous models (classical iso- thermal models or pressure/temperature unified models), machine learning models directly build data-driven models and avoid numerous theoretical assumptions and mathematical derivations.

Table 6 Evaluation metrics of different machine learning algorithms.

Training set Testing set

XGBoost ANN RF SVM XGBoost ANN RF SVM

MAE (scf/ton) 0.0068 0.0710 0.0291 0.0419 0.0358 0.1143 0.1607 0.1457 RMSE (scf/ton) 0.0001 0.0140 0.0032 0.0093 0.0036 0.0294 0.0631 0.0548 R2 0.9995 0.9384 0.9861 0.9592 0.9886 0.8983 0.7815 0.8101

Fig. 8. Performance of XGBoost model.

Table 7 Average evaluation metrics of different machine learning algorithms with five times optimization.

Training set Testing set

XGBoost ANN RF SVM XGBoost ANN RF SVM

MAE (scf/ton) 0.0051 0.2289 0.0135 0.0855 0.0404 0.3938 0.1662 0.2119 RMSE (scf/ton) 0.0001 0.0999 0.0013 0.0122 0.0053 0.3005 0.0601 0.1319 R2 0.9997 0.9648 0.9966 0.9755 0.9781 0.9186 0.9088 0.8411

M. Meng, et al. Fuel 278 (2020) 118358

9

Tang [5] mentioned three criteria for comparing adsorption models of methane in shale: the goodness-of-fit of the adsorption model for describing experimental raw data, the interpretation of the observed test phenomena, and the prediction capability of the adsorption models beyond the test data. The machine learning model proposed in this paper fits all of these three criteria. Our machine learning model could provide accurate predictions in training and testing data, and it pro- vides reasonable extrapolation. From the perspective of industrial ap- plications, these three criteria are not enough. First of all, for a new shale formation, if engineers need to estimate the adsorption behavior of the target formation using the classical models or pressure/tem- perature unified model, they still need to perform a time-consuming

and labor-intensive field coring, storage and transportation, and in-door adsorption testing. Second, the properties of individual shale in the target formation can be significantly different. The in-door tests may not represent the performance of the whole formation due to hetero- geneity. Using classical models, the studies are usually limited to aca- demic interest. Machine learning models give us the opportunity to significantly reduce the time spent on adsorption tests. Using all available historical experimental data, researchers only need to eval- uate some internal properties of the target shale formation, such as the TOC, and then they can estimate the adsorption of shale gas [7,42,43]. Chen [29] used geological parameters to estimate adsorbed gas content on a large scale, and he mentioned that adsorption experiments are still

(a) Extrapolation of temperature Case I (b) Extrapolation of temperature Case II

IIesaCCOTfonoitalopartxE)d(IesaCCOTfonoitalopartxE)c(

(e) Extrapolation of Moisture Case I (f) Extrapolation of Moisture case II Fig. 9. Extrapolation cases using trained XGBoost model.

M. Meng, et al. Fuel 278 (2020) 118358

10

necessary for the exact adsorbed gas content of a sample. With the machine learning model, researchers can avoid too many tedious ad- sorption tests and still achieve engineering-level accuracy for adsorp- tion contents. In the future, this machine learning model can be built into a large numerical frame to optimize the production curves of shale gas [38–41].

Note that the data used here is the excess adsorption amount, therefore, the predicted adsorption amount is also the excess adsorption amount. The absolute adsorption amount can be calculated based on the predicted excess adsorption amount. At the early stage, the excess adsorption curve and absolute adsorption curve overlap with each other, and the excess adsorption curve deviates from the absolute ad- sorption curve at a certain pressure with the increase of pore pressure. The following equation can be one way to calculate the absolute ad- sorption curve.

=n nabsolute a a freegas

e (13)

where a is the density of gas at adsorbed phase, = kg cm375a 3 for CH4 [44].

The above equation does not show the influence of inner chemical property on the relationship between absolute adsorption and excess adsorption. Some researchers [45] claim that the turning point could depend on the chemical property of given shale samples and differs for each different shale samples. If this factor is considered, we may need the chemical property of the shale samples to get the adsorption amount. But the machine learning method definitely saves considerable time and energy for excess adsorption tests, which benefits researchers.

5. Conclusion

The adsorption behavior of shale gas is a significant factor in the accurate assessment of natural gas adsorption under in situ conditions, which further influence the final determination of the geological GIP quantity and production life of shale gas wells. This paper compares the performance of methane adsorption behavior on shale among tradi- tional isothermal models, the temperature and pressure unified model, and machine learning models. From the above analysis, the following conclusions can be drawn:

(1) Eight traditional isothermal models are compared and analyzed with data from 20 cases. The modified DR models provide the best performance compared with Langmuir style models and Toth models. However, these models are limited to isothermal conditions and do not have the capability of extrapolating the predictions beyond the test condition. These models are also limited to a spe- cific type of shale.

(2) The temperature and pressure unified model has been proven to make a breakthrough in the isothermal conditions and it can ex- trapolate the predictions beyond the test ranges. However, it is still limited to a specific type of shale.

(3) Four machine learning algorithms (ANN, RF, SVM, XGBoost) are applied. Results show that XGBoost provides the optimal perfor- mance for both the training dataset and the testing dataset. This machine learning model can break the limit of the isothermal condition and shale type. It can also extrapolate the performance beyond the testing range. Furthermore, the machine learning model can be conveniently updated. If a new dataset is provided, the current model can be upgraded with a larger data set and provide broader applications.

(4) Machine learning models give us an opportunity to significantly reduce time-consuming and labor-intensive work. The machine learning model can be trained using all available historical ex- perimental data and updated whenever a new dataset is available. Using a well-trained machine learning model with a variety of

inputs, researchers only need to evaluate some internal properties of the target shale formation, such as the TOC, to simulate the adsorption behavior on shale. Furthermore, this machine learning model can be built into a large numerical frame to optimize the production curves of shale gas.

CRediT authorship contribution statement

Meng Meng: Conceptualization, Methodology, Software, Investigation, Writing - original draft, Writing - review & editing. Ruizhi Zhong: Software, Validation, Writing - review & editing. Zhili Wei: Software.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influ- ence the work reported in this paper.

Acknowledgment

The author(s) received no financial support for the research, au- thorship, and/or publication of this article.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https:// doi.org/10.1016/j.fuel.2020.118358.

References

[1] Today in Energy, 2019. U.S. natural gas production hit a new record high in 2018. https://www.eia.gov/todayinenergy/detail.php?id=38692.

[2] Trent Jacobs China flexes shale muscles as CNPC ups gas output 40% Journal of Petroleum Technology. 2019 https://www.spe.org/en/jpt/jpt-article-detail/ ?art=4955.

[3] Tang X, Ripepi N, Stadie NP, et al. A dual-site Langmuir equation for accurate es- timation of high pressure deep shale gas resources. Fuel 2016;185:10–7.

[4] Tan J, Weniger P, Krooss B, et al. Shale gas potential of the major marine shale formations in the Upper Yangtze Platform, South China, Part II: Methane sorption capacity. Fuel 2014;129:204–18.

[5] Tang X, Ripepi N, Luxbacher K, et al. Adsorption models for methane in shales: Review, comparison, and application. Energ Fuel 2017;31:10787–801.

[6] Jiang Z, Zhao L, Zhang D. Study of adsorption behavior in shale reservoirs under high pressure. J Nat Gas Sci Eng 2018;49:275–85.

[7] Li J, Chen Z, Wu K, et al. A multi-site model to determine supercritical methane adsorption in energetically heterogeneous shales. Chem Eng J 2018;349:438–55.

[8] Chen L, Zuo L, Jiang Z, et al. Mechanisms of shale gas adsorption: Evidence from thermodynamics and kinetics study of methane adsorption on shale. Chem Eng J 2019;361:559–70.

[9] Weniger P, Kalkreuth W, Busch A, et al. High-pressure methane and carbon dioxide sorption on coal and shale samples from the Paraná Basin. Brazil Int J Coal Geol 2010;84:190–205.

[10] Beaton AP, Pawlowicz JG, Anderson SDA, et al. Rock Eval™, total organic carbon, isotherms and organic petrography of the Colorado Group: shale gas data release; Energy Resources Conservation Board, ERCB/AGS Open File Report 2008-11 (2009): 88.

[11] Beaton AP, Pawlowicz JG, Anderson SDA, et al. Rock Eval™, total organic carbon and adsorption isotherms of the Duvernay and Muskwa formations in Alberta: shale gas data release; Energy Resources Conservation Board, ERCB/AGS Open File Report 2010-04 (2010): 33.

[12] Beaton AP, Pawlowicz JG, Anderson SDA, et al. Rock Eval™, total organic carbon and adsorption isotherms of the Montney Formation in Alberta: shale gas data re- lease; Energy Resources Conservation Board, ERCB/AGS Open File Report 2010-05 (2010): 37.

[13] Dubinin MM, Radushkevich LV. The equation of the characteristic curve of the activated charcoal. Proc Union Soviet Soc Rep Acad Sci 1947;55:331–7.

[14] Sakurovs R, Day S, Weir S, Duffy G. Application of a modified Dubinin− Radushkevich equation to adsorption of gases by coals under supercritical condi- tions. Energy Fuels 2007;21(2):992–7.

[15] Hutson ND, Yang RT. Theoretical basis for the Dubinin-Radushkevitch (DR) ad- sorption isotherm equation. Adsorption 1997;3:189–95.

[16] Zhou S, Xue H, Ning Y, et al. Experimental study of supercritical methane ad- sorption in Longmaxi shale: insights into the density of adsorbed methane. Fuel 2018;211:140–8.

[17] Song X, Lü X, Shen Y, et al. A modified supercritical Dubinin-Radushkevich model

M. Meng, et al. Fuel 278 (2020) 118358

11

for the accurate estimation of high pressure methane adsorption on shales. Int J Coal Geol 2018;193:1–15.

[18] Butt HJ, Graf K, Kappl M. Physics and chemistry of interfaces. John Wiley & Sons; 2013.

[19] Masel RI. Principles of adsorption and reaction on solid surfaces. John Wiley & Sons; 1996.

[20] Abousleiman Y, Nguyen V. Poromechanics response of inclined wellbore geometry in fractured porous media. J Eng Mech 2005;131(11):1170–83.

[21] Zhang J, Bai M, Roegiers JC. Dual-porosity poroelastic analyses of wellbore stabi- lity. Inter J Rock Mech Min Sci 2003;40(4):473–83.

[22] Meng M, Baldino S, Miska S. etc. Wellbore stability in naturally fractured forma- tions featuring dual-porosity/single-permeability and finite radial fluid discharge. J Petrol Sci Eng 2019;174:790–803.

[23] Boyd S, Parikh N, Chu E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends® Mach Learn 2011;3(1):1–122.

[24] Huang G, Huang GB, Song S, You K. Trends in extreme learning machines: a review. Neural Netw 2015;61:32–48.

[25] Huang GB, Zhou H, Ding X, et al. Extreme learning machine for regression and multiclass classification. IEEE Tran Syst, Man, Cybern, Part B (Cybern) 2012;42(2):513–29.

[26] Liaw A, Wiener M. Classification and regression by randomForest. R news 2002;2(3):18–22.

[27] Rasmussen CE. Gaussian processes in machine learning. Summer School on Machine Learnin. Heidelberg: Springer Berlin; 2003.

[28] Meng M, Qiu Z, Zhong R, et al. Adsorption characteristics of supercritical CO2/CH4 on different types of coal and a machine learning approach. Chem Eng J 2019;368C:847–64.

[29] Chen Y, Jiang S, Zhang D, et al. An adsorbed gas estimation model for shale gas reservoirs via statistical learning. Appl Energ 2017;197:327–41.

[30] Xiong W, Zuo L, Luo L, et al. Methane adsorption on shale under high temperature and high pressure of reservoir condition: Experiments and supercritical adsorption modeling. Adsorpt Sci Technol 2016;34(2–3):193–211.

[31] Li J, Zhou S, Gaus G, et al. Characterization of methane adsorption on shale and

isolated kerogen from the Sichuan Basin under pressure up to 60 MPa: Experimental results and geological implications. Int J Coal Geol 2018;189:83–93.

[32] Zhao P, Mao Z, Huang Z, Zhang C. A new method for estimating total organic carbon content from well logs. AAPG Bull 2016;100(8):1311–27.

[33] Chen T, Guestrin C. Xgboost: A scalable tree boosting system. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. ACM; 2016.

[34] Brownlee J, XGBoost with Python. Machine Learning with Mastery; 2016. [35] Breiman L. Random forests. Machine learning 2001;45(1):5–32. [36] Steinwart I, Christmann A. Support vector machines. Springer Science & Business

Media; 2008. [37] Géron A. Hands-on machine learning with scikit-learn, keras, and tensorflow:

concepts, tools, and techniques to build intelligent systems. O'Reilly Media; 2019. [38] Chen B, Harp DR, Lin Y, et al. Geologic CO2 sequestration monitoring design: a

machine learning and uncertainty quantification based approach. Appl Energ 2018;225:332–45.

[39] Cui G, Wang Y, Rui Z, et al. Assessing the combined influence of fluid-rock inter- actions on reservoir properties and injectivity during CO2 storage in saline aquifers. Energ 2018;155:281–96.

[40] Zhang Y, Lebedev M, Jing Y, et al. In-situ X-ray micro-computed tomography imaging of the microstructural changes in water-bearing medium rank coal by su- percritical CO2 flooding. Inter J Coal Geolog 2019;203:28–35.

[41] Lebedev M, Zhang Y, Sarmadivaleh M, et al. Carbon geosequestration in limestone: pore-scale dissolution and geomechanical weakening. Int J Greenh Gas Con 2017;66:106–19.

[42] Wang S, Feng Q, Javadpour F, et al. Competitive adsorption of methane and ethane in montmorillonite nanopores of shale at supercritical conditions: a grand canonical Monte Carlo simulation study. Chem Eng J 2019;355:76–90.

[43] Li J, Wu K, Chen Z, Wang W, Yang B, Wang K, et al. Effects of energetic hetero- geneity on gas adsorption and gas storage in geologic shale systems. Appl Energ 2019;251:113368.

[44] Zhang C, Zhang J. Coal Chemistry. Coal Industry Press; 2012. [45] Tian Y, Yan C, Jin Z. Characterization of methane excess and absolute adsorption in

various clay nanopores from molecular simulation. Sci Rep 2017;7(1):12040.

M. Meng, et al. Fuel 278 (2020) 118358

12