gRPC server mock is done.

README.md

@@ -69,16 +69,18 @@ English, please click [here](./HELP.md)
 ```
 #### 1.1.3 构造微服务 mock server，根据不同的请求返回不同的响应指定结果 (集成测试)  
 ##### 测试方法  
-```text
-1.  准备json格式的请求响应数据；  
-2.  不同于 1.1.1 和 1.1.2 部分，mock server将无法集成到一个具体的测试用例中，它应该被部署为一个真实服务供客户端服务(在开发中的测试服务对象)调用；  
-3.  启动命令：java -jar auto-test-1.0.0-SNAPSHOT.jar --mvcmock.response.path=<> --server.ssl.enabled=<>  
-    参数1：请求响应数据路径，默认为./mvcmock/<serviceName.json>
-    参数2：是否启用https，默认为false。
-4.  工作原理说明：
-    - mock server将逐一匹配请求数据的每一个域，最后返回与请求数据匹配度最高的某一个测试数据中response域的值。
+```html
+1.  不同于 1.1.1 和 1.1.2 部分，mock server将无法集成到一个具体的测试用例中，它应该被部署为一个真实服务供客户端服务(在开发中的测试服务对象)调用； 
+2.  若决定是模拟单独一个服务，则修改[application.yml](./src/main/resources/application.yml)中参数 serviceMock.mocker，赋值为某一个具体的服务名称；  
+    若希望在一个服务器中模拟所有服务，则修改上述参数的值为allRestInOne；
+3.  准备json格式的请求和响应数据，存放在servicemocks目录；  
+4.  执行命令行命令 gradlew build，将生成的 jar 拷贝到目的服务器，或者通过 Dockerfile 生成镜像；
+5.  启动命令：java -jar auto-test-1.0.0-SNAPSHOT.jar --server.ssl.enabled=<>      
+    参数：是否启用https，默认为false。
+6.  工作原理说明：
+    - mock server将逐一匹配请求数据的每一个域，最后返回与请求数据匹配度最高的某个测试数据中response域的值。
     - 即，不同的两个测试数据，可以仅有一个域的值不同；mock server进行精确匹配，返回匹配成功的那个数据的response域值。
-    - [数据格式参考](./mvcmock/backend.json)
+    - [数据格式参考](./servicemocks/backend.json)
 ```
 ##### 用例参考  
 ```
@@ -93,6 +95,10 @@ N/A
    - 相关参数应根据在application.properties内的定义进行修改  
    - curl命令验证例子: curl -H "Content-Type:application/json" -X POST -d '{"user_id": "123", "coin":100, "success":1, "msg":"OK!" }' -v --http2-prior-knowledge http://172.22.17.74:8080/userservice/fedFromOrg/87654321  
 ```
+##### 有待改进
+```text
+1.  需要支持插件等形式，对响应数据进行用户/请求差别化地动态修改（如，变量替换，实时算法调用等）
+```
 
 ### 1.2 RPC API的自动化测试  
 ```text
@@ -120,7 +126,7 @@ RPC形式的API，又以 gRPC 和 json-RPC 两种实现方式居多。
 2.  通过 gradle 插件命令生成java class。可携带参数chainVer，表示proto文件版本。  
     -  gradlew generateProto -DchainVer=v1.6.6  
 3.  编写 gRPC 客户端请求数据文件([参考链接](/src/test/resources/testdatacollection/v2.2.0/runtime/chain_ops/sendTransaction.json))  
-3.  由于 gRPC 接口对负载净荷数据没有统一规范要求，所以只能手工实现请求数据的发送和响应数据的验证：  
+3.  由于 gRPC 接口对负载净荷数据有其自定义的规范要求，所以只能手工实现请求数据的类型转换，然后发送和响应数据的验证：  
     -  编写 gRPC 客户端发送程序代码(主要是把步骤3中的数据，序列化成2进制码流。[参考链接](/src/main/java/com/fuzamei/autotest/rpc/grpc/GrpcClientTransaction.java))  
     -  编写 gRPC 服务端响应数据验证代码(主要是和 proto 文件中定义的 service.returns 进行结果比对)。  
 ```
@@ -132,10 +138,11 @@ RPC形式的API，又以 gRPC 和 json-RPC 两种实现方式居多。
 ```text
 - N/A
 ```
-##### 有待提高
+##### 有待改进
 ```text
-- 对于语法不完整的 proto 文件，无法实现自动化验证；  
-- 相对 RESTful 接口自动化测试用例来说，RPC接口自动化测试用例需要更多的代码编写量。
+-  对于语法不完整的 proto 文件，无法实现自动化验证；  
+-  相对 RESTful 接口自动化测试用例来说，RPC接口自动化测试用例需要更多的代码编写量；  
+-  需要找寻一个通用的解决方案去实现：测试数据与实际发送数据的自动类型转换机制。
 ```
   
 #### 1.2.3 适用于 json-RPC API 的 mock server [参考1.1.3]  

build.gradle

@@ -43,6 +43,7 @@ dependencies {
 	implementation 'com.google.protobuf:protobuf-java-util:3.17.3'
 	//implementation 'io.grpc:grpc-netty:1.39.0'
 	implementation 'net.devh:grpc-client-spring-boot-starter:2.12.0.RELEASE' exclude group: 'io.grpc', module: 'grpc-netty-shaded'
+	implementation 'net.devh:grpc-server-spring-boot-starter:2.12.0.RELEASE'
 
 	implementation 'com.alibaba:fastjson:1.2.47'
 	implementation 'org.bitcoinj:bitcoinj-core:0.14.7'

servicemocks/chain33.json

+{
+  "GetBlocks.001": {
+    "description": "节点名必须是实际可调用方法名做前缀",
+    "request": {},
+    "response": {}
+  },
+  "getLastHeader.001": {
+    "description": "模拟chain33返回请求接口getLastHeader进行的响应。",
+    "request": {
+    },
+    "response": {
+      "version": 0,
+      "parentHash": "0xc39daaba9f6ba5b9b27de3c17cc5d197c2501bcdc55461ce101e2017d1c62e47",
+      "txHash": "0x671228c599fb6e252ba0dc7f26687d44249898d90cd8f1609d7820458550d4cd",
+      "stateHash": "0xdd0bdee3ab1416e5a5798293137650b4f0294f9aaadc241881af01707b116b35",
+      "height": 21,
+      "blockTime": 1628759513,
+      "txCount": 1,
+      "hash": "0x65e64018a8ad0338bf3a5fdddde01680e918940a89eef6143e9cb8e7c4c255f7",
+      "difficulty": 520159231
+    }
+  }
+}
\ No newline at end of file

src/main/java/com/fuzamei/autotest/properties/MockServiceProperties.java

+package com.fuzamei.autotest.properties;
+
+import lombok.Getter;
+import lombok.Setter;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.context.annotation.PropertySource;
+import org.springframework.stereotype.Component;
+
+@Component
+@PropertySource(value = "classpath:application.yml", encoding = "UTF-8")
+@Setter
+@Getter
+public class MockServiceProperties {
+
+    @Value("${serviceMock.mocker}")
+    private String serviceMocker;
+
+    @Value("${serviceMock.allRestInOne.responsePath}")
+    private String mockAllRestResponsePath;
+
+    @Value("${serviceMock.backend.responsePath}")
+    private String mockBackendResponsePath;
+
+    @Value("${serviceMock.chain33.responsePath}")
+    private String mockChain33ResponsePath;
+}

src/main/java/com/fuzamei/autotest/properties/SpecificProperties.java

@@ -23,7 +23,4 @@ public class SpecificProperties {
 
     @Value("${restassured.sockettimeout}")
     private Integer restSocketTimeout;
-
-    @Value("${mvcmock.response.path}")
-    private String mvcMockResponsePath;
 }

src/main/java/com/fuzamei/autotest/rpc/grpc/GrpcClientTransaction.java → src/main/java/com/fuzamei/autotest/rpc/grpc/GrpcClients.java

@@ -7,6 +7,7 @@ import io.grpc.StatusRuntimeException;
 import lombok.extern.slf4j.Slf4j;
 import net.devh.boot.grpc.client.inject.GrpcClient;
 import org.springframework.stereotype.Service;
+import types.BlockChainProto;
 import types.CommonProto;
 import types.TransactionProto;
 import types.chain33Grpc.*;
@@ -16,9 +17,9 @@ import static org.hamcrest.Matchers.equalTo;
 
 @Service
 @Slf4j
-public class GrpcClientTransaction {
+public class GrpcClients {
 
-    @GrpcClient("chain33")
+    @net.devh.boot.grpc.client.inject.GrpcClient("chain33")
     private chain33BlockingStub stub;
 
     public void sendMessageToSendTransaction(JsonNode transaction, JsonNode returns) {
@@ -38,14 +39,25 @@ public class GrpcClientTransaction {
                     .setHeader(ByteString.copyFromUtf8(transaction.get("header").asText()))
                     .setNext(ByteString.copyFromUtf8(transaction.get("next").asText()))
                     .setChainID(transaction.get("chainID").asInt()).build());
-            log.debug("gRPC response is: {}", response.getMsg());
+            log.debug("gRPC response is: {}", response);
             //JsonFormat.parser().ignoringUnknownFields().merge(json, yourObjectBuilder);
 
         }
         catch (final StatusRuntimeException e) {
             log.error("failed to send message to call gRPC interface as error: {}", e);
-            assertThat("failed to send message to call gRPC Register interface.", true, equalTo(false));
+            assertThat("failed to send message to call gRPC Register interface.", e, equalTo(null));
         }
 
     }
+
+    public void getLastHeader() {
+        try {
+            BlockChainProto.Header header = this.stub.getLastHeader(CommonProto.ReqNil.newBuilder().build());
+            log.debug("gRPC response is: {}", header);
+        }
+        catch (final StatusRuntimeException e) {
+            log.error("failed to send message to call gRPC interface as error: {}", e);
+            assertThat("failed to send message to call gRPC Register interface.", e, equalTo(null));
+        }
+    }
 }

src/main/java/com/fuzamei/autotest/servicemock/grpc/GrpcServerService.java

+package com.fuzamei.autotest.servicemock.grpc;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fuzamei.autotest.properties.MockServiceProperties;
+import com.google.protobuf.ByteString;
+import lombok.extern.slf4j.Slf4j;
+import net.devh.boot.grpc.server.service.GrpcService;
+import org.springframework.beans.factory.annotation.Autowired;
+import types.BlockChainProto;
+import types.chain33Grpc;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.Iterator;
+import java.util.Map;
+
+@Slf4j
+@GrpcService
+public class GrpcServerService extends chain33Grpc.chain33ImplBase {
+
+    @Autowired
+    MockServiceProperties mockServiceProperties;
+
+    @Override
+    public void getLastHeader(types.CommonProto.ReqNil request,
+                              io.grpc.stub.StreamObserver<types.BlockChainProto.Header> responseObserver) {
+        JsonNode rootNode = null;
+        JsonNode testDataNode = null;
+        try{
+            File file = new File(mockServiceProperties.getMockChain33ResponsePath());
+            ObjectMapper objectMapper = new ObjectMapper();
+            rootNode = objectMapper.readTree(file);
+        }
+        catch (IOException exception) {
+            log.error("Data file is not existed or it is not a valid json file.", exception);
+            return;
+        }
+        Iterator<Map.Entry<String, JsonNode>> it = rootNode.fields();
+        while (it.hasNext()) {
+            Map.Entry<String, JsonNode> eachDataNode = it.next();
+            if (eachDataNode.getKey().split("\\.")[0].equals("getLastHeader")) {
+                testDataNode = eachDataNode.getValue().get("response");
+                break;
+            }
+        }
+        BlockChainProto.Header header = BlockChainProto.Header.newBuilder()
+                .setVersion(testDataNode.get("version").asLong())
+                .setParentHash(ByteString.copyFromUtf8(testDataNode.get("parentHash").asText()))
+                .setTxHash(ByteString.copyFromUtf8(testDataNode.get("txHash").asText()))
+                .setStateHash(ByteString.copyFromUtf8(testDataNode.get("stateHash").asText()))
+                .setHeight(testDataNode.get("height").asLong())
+                .setBlockTime(testDataNode.get("blockTime").asLong())
+                .setTxCount(testDataNode.get("txCount").asLong())
+                .setHash(ByteString.copyFromUtf8(testDataNode.get("hash").asText()))
+                .setDifficulty(testDataNode.get("difficulty").asInt())
+                .build();
+        responseObserver.onNext(header);
+        responseObserver.onCompleted();
+    }
+
+}

src/main/java/com/fuzamei/autotest/servicemock/RestEntity.java → src/main/java/com/fuzamei/autotest/servicemock/rest/RestEntity.java

-package com.fuzamei.autotest.servicemock;
+package com.fuzamei.autotest.servicemock.rest;
 
 import com.fasterxml.jackson.databind.JsonNode;
 import lombok.AllArgsConstructor;

src/main/java/com/fuzamei/autotest/servicemock/RestMockController.java → src/main/java/com/fuzamei/autotest/servicemock/rest/RestMockController.java

-package com.fuzamei.autotest.servicemock;
+package com.fuzamei.autotest.servicemock.rest;
 
 import com.fasterxml.jackson.databind.JsonNode;
 import lombok.extern.slf4j.Slf4j;

src/main/java/com/fuzamei/autotest/servicemock/RestMockService.java → src/main/java/com/fuzamei/autotest/servicemock/rest/RestMockService.java

-package com.fuzamei.autotest.servicemock;
+package com.fuzamei.autotest.servicemock.rest;
 
-import com.fasterxml.jackson.core.JsonProcessingException;
 import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fuzamei.autotest.properties.MockServiceProperties;
 import com.fuzamei.autotest.properties.SpecificProperties;
+import com.fuzamei.autotest.servicemock.rest.RestEntity;
 import lombok.extern.slf4j.Slf4j;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
@@ -18,21 +19,32 @@ import java.util.*;
 public class RestMockService {
 
     @Autowired
-    SpecificProperties specificProperties;
+    MockServiceProperties mockServiceProperties;
 
     public Map<String, Object> respondAsDefinedJsonFile(RestEntity request) {
         Map<String, Object> mapResp = new HashMap<>();
         JsonNode response = null;
         JsonNode rootNode = null;
         String failureCause = "Cannot find out a valid response to match request. Please check your testing data and request message.";
+        String serviceMockFilePath = null;
         try{
-            File file = new File(specificProperties.getMvcMockResponsePath());
+            switch (mockServiceProperties.getServiceMocker().toLowerCase(Locale.ROOT)){
+                case "backend":
+                    serviceMockFilePath = mockServiceProperties.getMockBackendResponsePath();
+                    break;
+                case "allrestinone":
+                    serviceMockFilePath = mockServiceProperties.getMockAllRestResponsePath();
+                    break;
+                default:
+                    return mapResp;
+            }
+            File file = new File(serviceMockFilePath);
             ObjectMapper objectMapper = new ObjectMapper();
             rootNode = objectMapper.readTree(file);
         }
         catch (IOException exception) {
             log.error("Data file is not existed or it is not a valid json file.", exception);
-            failureCause = "The data json file [" + specificProperties.getMvcMockResponsePath() + "] is not existed or it is not a valid json file";
+            failureCause = "The data json file [" + serviceMockFilePath + "] is not existed or it is not a valid json file";
             mapResp.put("response", null);
             mapResp.put("failureCause", failureCause);
             return mapResp;

src/main/java/com/fuzamei/autotest/rpc/grpc/GrpcClientLogInterceptor.java → src/main/java/com/fuzamei/autotest/utils/GrpcClientLogInterceptor.java

-package com.fuzamei.autotest.rpc.grpc;
+package com.fuzamei.autotest.utils;
 
 
 import io.grpc.*;

src/main/java/com/fuzamei/autotest/utils/GrpcLogInterceptorConfiguration.java

+package com.fuzamei.autotest.utils;
+
+import net.devh.boot.grpc.server.interceptor.GrpcGlobalServerInterceptor;
+import org.springframework.context.annotation.Configuration;
+import org.springframework.core.Ordered;
+import org.springframework.core.annotation.Order;
+
+import net.devh.boot.grpc.client.interceptor.GrpcGlobalClientInterceptor;
+
+@Order(Ordered.LOWEST_PRECEDENCE)
+@Configuration(proxyBeanMethods = false)
+public class GrpcLogInterceptorConfiguration {
+
+    @GrpcGlobalClientInterceptor
+    GrpcClientLogInterceptor logClientInterceptor() {
+        return new GrpcClientLogInterceptor();
+    }
+
+    @GrpcGlobalServerInterceptor
+    GrpcServerLogInterceptor logServerInterceptor() {
+        return new GrpcServerLogInterceptor();
+    }
+
+}

src/main/java/com/fuzamei/autotest/utils/GrpcServerLogInterceptor.java

+package com.fuzamei.autotest.utils;
+
+import io.grpc.*;
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+public class GrpcServerLogInterceptor implements ServerInterceptor {
+    @Override
+    public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> serverCall, Metadata metadata,
+                                                                 ServerCallHandler<ReqT, RespT> serverCallHandler) {
+        log.info("now is calling gRPC method: {}", serverCall.getMethodDescriptor().getFullMethodName());
+        return serverCallHandler.startCall(serverCall, metadata);
+    }
+}

src/main/resources/application.properties

@@ -23,6 +23,7 @@ logging.level.org.springframework=ERROR
 logging.level.com.fuzamei.autotest=DEBUG
 logging.config=classpath:logback-spring.xml
 spring.jackson.default-property-inclusion=non_null
+spring.application.name=auto-test
 
 mybatis.type-aliases-package=com.fuzamei.autotest.testdata.entity
 mybatis.mapper-locations=classpath:mybatis/mapper/*.xml
@@ -34,7 +35,4 @@ restassured.sockettimeout=10000
 okhttp3.http2.maxtotalconnections=10
 okhttp3.http2.connectionkeepaliveduration=10
 okhttp3.http2.calltimeout=10
-okhttp3.http2.connectiontimeout=10
-
-mvcmock.response.path=mvcmock/backend.json
-mvcmock.https=false
\ No newline at end of file
+okhttp3.http2.connectiontimeout=10
\ No newline at end of file

src/main/resources/application.yml

 grpc:
   client:
     chain33:
-      address: static://172.22.19.2:8802
+      address: static://127.0.0.1:8802
       enableKeepAlive: true
       keepAliveWithoutCalls: true
       negotiationType: plaintext
+  server:
+    port: 8802
 service:
   backend:
     host: 172.22.18.152
@@ -27,3 +29,13 @@ spring:
     password: baas
     url: jdbc:mysql://${service.mysql.host}:3306/${service.mysql.database}?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8
     driver-class-name: com.mysql.cj.jdbc.Driver
+
+serviceMock:
+  mocker: chain33
+  pathPrefix: servicemocks
+  allRestInOne:
+    responsePath: ${serviceMock.pathPrefix}/allrest.json
+  backend:
+    responsePath: ${serviceMock.pathPrefix}/backend.json
+  chain33:
+    responsePath: ${serviceMock.pathPrefix}/chain33.json

src/test/java/com/fuzamei/autotest/steps/rpc/GrpcRequestVerification.java

@@ -5,7 +5,7 @@ import com.fasterxml.jackson.core.JsonProcessingException;
 import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
 import com.fuzamei.autotest.properties.GlobalProperties;
-import com.fuzamei.autotest.rpc.grpc.GrpcClientTransaction;
+import com.fuzamei.autotest.rpc.grpc.GrpcClients;
 import io.cucumber.java.en.Then;
 import lombok.extern.slf4j.Slf4j;
 import org.springframework.beans.factory.annotation.Autowired;
@@ -15,7 +15,7 @@ import java.io.File;
 @Slf4j
 public class GrpcRequestVerification {
     @Autowired
-    private GrpcClientTransaction grpcClientTransaction;
+    private GrpcClients grpcClients;
 
     @Autowired
     private GlobalProperties globalProperties;
@@ -30,7 +30,7 @@ public class GrpcRequestVerification {
             JsonNode expectedRespDataNode = dataNode.get("result");
             switch (rpcType){
                 case "gRPC":
-                    grpcClientTransaction.sendMessageToSendTransaction(reqDataNode, expectedRespDataNode);
+                    grpcClients.sendMessageToSendTransaction(reqDataNode, expectedRespDataNode);
                     break;
                 case "jRPC":
                     break;
@@ -41,4 +41,15 @@ public class GrpcRequestVerification {
             e.printStackTrace();
         }
     }
+
+    @Then("^Get the last block header in chain33 by (.*?) interface$")
+    public void getLastHeaderByRpcInterface(String rpcType) throws Throwable {
+        switch (rpcType){
+            case "gRPC":
+                grpcClients.getLastHeader();
+                break;
+            case "jRPC":
+                break;
+        }
+    }
 }

src/test/resources/features/apitest/grpc/accountManager.feature

 @rpcTry
-Feature: Account Management Contract Test
+Feature: gRPC Interface Dry Run Test
 
-  Scenario: Account registration by RPC interface
+  Scenario: Send a transaction to server by gRPC interface
     Given The testing RUNTIME data chain_ops.sendTransaction is ready
-    Then Register an account in chain33 with dataNum sendTransaction.001 by gRPC interface
\ No newline at end of file
+    Then Register an account in chain33 with dataNum sendTransaction.001 by gRPC interface
+
+  Scenario: Request to server with null data by gRPC interface
+    Then Get the last block header in chain33 by gRPC interface
\ No newline at end of file