On cloudbase + cloud function / cloud hosting

Lupoy 2021-09-15 10:21:40

1311631169644_.pic.jpg

A glass of wine , Jianghu night rain ten years lamp .

Preface

Using cloud development cloudbase When , It always comes to mind from time to time : How to separate back-end operations such as calling database from front-end business when using cloud development ?

Reading. 「CloudBase CMS」 Source code , I sprouted the use of nestjs+cloudbase+ Cloud functions ( No mistake. , The first thing I want to play is cloud function ) Ideas .

nestjs What is it? ?

image.png

The document describes it like this :

Nest It's a tool for building efficient , Extensible  Node.js  The framework for server-side applications . It uses progressive JavaScript, Built in and fully supported  TypeScript( But developers are still allowed to use pure JavaScript Write code ) And combined OOP( object-oriented programming ),FP( Functional programming ) and FRP( Functional response programming ) The elements of . In the underlying ,Nest Use powerful HTTP Server frame , Such as Express( Default ) and Fastify.Nest On top of these frameworks there is a certain level of abstraction , At the same time, it will be API Direct exposure to developers . This makes it easy to use countless third-party modules on each platform .

cloudbase What is it again? ?

Development of cloud (Tencent CloudBase,TCB) It is the cloud native integrated development environment and tool platform provided by Tencent cloud , High availability for developers 、 Automatic elastic scaling back-end cloud services , Include calculations 、 Storage 、 Hosting etc. serverless Chemical ability , It can be used for cloud integrated development of multiple end applications ( Applet 、 official account 、Web application 、Flutter Client, etc ), Help developers build and manage back-end services and cloud resources in a unified way , It avoids the tedious server construction and operation and maintenance in the application development process , Developers can focus on the implementation of business logic , The development threshold is lower , More efficient .

Why is there a cloud hosting ?

Cloud hosting (Tencent CloudBase Run) It's cloud development (Tencent CloudBase,TCB) Provides a new generation of cloud native application engine (App Engine 2.0), Support hosting container applications written in any language and framework . Develop other products with the cloud ( Cloud functions 、 Cloud database 、 Cloud storage 、 Expand applications 、HTTP Access the service 、 Static website hosting, etc ) Provide users with cloud native integrated development environment and tool platform , High availability for developers 、 Automatic elastic scaling back-end cloud services , It can be used for cloud integrated development of multiple end applications ( Applet 、 official account 、Web application 、 Microservice application 、Flutter Client, etc ), Avoid tedious server construction and operation and maintenance in the process of application development , So that developers can focus on the implementation of business logic , The development threshold is lower , More efficient .

be based on nestjs Build the project

because cloudbase Provides nestjs Framework , So we can use cloudbase Scaffold tools for construction , By default, we have a cloud development environment , And the cloud development environment has been configured on Tencent cloud console .

Scaffolding creation project

  • Install scaffolding tools
npm i -g @cloudbase/cli
 Copy code 
  • Test the installation for success

cloudbase After the scaffold is installed successfully , We can use cloudbase -v Check to see if the installation is successful , Of course , For easy input ,cloudbase Or we could just write it as tcb

tcb -v
 Copy code 
  • land cloudbase
cloudbase login
 Copy code 
  • Create projects locally
tcb new <appName> [template]
# such as 
tcb new nest_test nest-starter
 Copy code 

Project documents

file effect
app.controller.ts An example of a basic controller with a single route .
app.controller.spec.ts For the unit test example of the basic controller
app.module.ts The root module of the application .
app.service.ts Basic services with a single method
main.ts Application entry file . It USES NestFactory Used to create Nest Application example .

Cloud function construction

In the beginning , The project is deployed in the form of cloud functions , Then you must configure cloudbaserc.json file :

{
"version": "2.0",
"envId": " Environmental Science ID",
"$schema": "https://framework-1258016615.tcloudbaseapp.com/schema/latest.json",
"framework": {
"name": " Function name ",
"plugins": {
"node": {
"use": "@cloudbase/framework-plugin-node",
"inputs": {
"name": " Function name ",
"path": "/ Function path ",
"entry": "app.js",
// Configuration function build command 
"buildCommand": "npm install --prefer-offline --no-audit --progress=false && npm run build",
"functionOptions": {
"timeout": 5,
// environment variable 
"envVariables": {
"NODE_ENV": "production",
"TCB_ENV_ID": "{{env.TCB_ENV_ID}}",
}
}
}
}
}
}
}
 Copy code 

cloudbaserc.json The file will be automatically recognized .env.local or .env Environment variables in , Therefore, we can dynamically introduce the configured environment variables in the form of curly braces , meanwhile , In order to prevent the upload of sensitive information , We usually stay in .gitignore Configuration in file : prohibit .env.local or .env Upload to warehouse .

# compiled output
/dist
/node_modules
.env.local
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*
# OS
.DS_Store
# Tests
/coverage
/.nyc_output
# IDEs and editors
/.idea
.project
.classpath
.c9/
*.launch
.settings/
*.sublime-workspace
# IDE - VSCode
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
 Copy code 

The difficulty of uploading cloud functions is simple , So when we finish configuring and uploading , We can pass... In the project app.callFunction() In the form of .

Problems encountered with cloud functions

Because I need to provide a stingy API Interface , But I will base64 An error was reported when uploading the picture :EXCEED_MAX_PAYLOAD_SIZE Query the document 」 hear :

HTTP The request body exceeds the maximum volume limit ( Text 100 KB, Binary system 20MB)

The first thought

The initial idea was that we set up... In the project body Limited volume of , To solve this problem , But after setting, it is found that it cannot be changed directly http The size of the request body , By querying logs in Tencent cloud console, you can find , Each request cannot enter the actual code , Therefore, we cannot configure the size of the request body .

By contacting cloudbase Development students know : Cloud functions are restricted by inclusions , And provides two solutions :

  1. adopt @cloudbae/js-sdk Client package base64 Upload your pictures , After through fileId Pass it to the back end for human analysis , But we don't intend to store images uploaded by users , Therefore, this scheme is temporarily excluded

  2. Deploy in the form of cloud hosting , Mount the service , In this way, you can set the size of the request question .

Pass ratio pair , Therefore, I decided to deploy services in the form of cloud hosting .

The difference between cloud functions and cloud hosting

modular Cloud functions Cloud hosting
Request concurrency Single instance single concurrency , When multiple instances are concurrent, multiple instances need to be pulled up for processing Multiple concurrent instances
Language / frame Development language and framework support is limited Compatible with existing frameworks
Problem location Easy to locate Relatively flexible , Rely on customization
Resident operation I won't support it Support
Log monitoring Based on functions Based on services
Version grayscale Support by traffic grayscale Support by traffic grayscale 、 Press URL Parameter grayscale
Elastic expansion and contraction Support Support
Foreign service Provide default URL and SDK Provide default URL
Cross platform Function rules are different , Difficult to deploy across platforms Can be deployed across platforms
Private deployment I won't support it Portable privatization / Mixed deployment
It's hard to get started Simple secondary
Billing method Charge per request 、 By the number of requests and by each call GBS Consumed by container operation CPU、 Memory 、 External network traffic generated by service 、 Service build time

Cloud hosting build

The definition of cloud hosting has been mentioned above . There will be no more details here . Interested students can refer to 「 Cloud hosted documents 」.

How to quickly build a cloud hosting Nodejs application ?

Apply containerization

Under the project root , Create a file called  Dockerfile  The file of , The contents are as follows :

# Official use Node.js 12 Lightweight mirroring .
# https://hub.docker.com/_/node
FROM node:12-slim
# Define working directory 
WORKDIR /usr/src/app
# Copy the dependency definition file to the working directory 
COPY package*.json ./
# With production Form installation depends on 
RUN npm install --only=production
# Copy the local code to the working directory 
COPY . ./
# Start the service 
CMD [ "node", "index.js" ]
 Copy code 

that , How to build a cloud hosted Nestjs project ?

Based on the above dockerfile The file of , We can make relevant configurations :

# Official use Node.js 12 Lightweight mirroring 
FROM node:12.15.0-alpine
# Set up Alpine linux System time zone 
RUN apk --update add tzdata \ && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \ && echo "Asia/Shanghai" > /etc/timezone \ && apk del 
RUN mkdir -p /usr/src/app
# Define working directory 
WORKDIR /usr/src/app
# Copy the dependency definition file to the working directory 
COPY package.json /usr/src/app/package.json
# Copy the dependency definition file to the working directory 
COPY package*.json ./
# Copy the local code to the working directory 
COPY . ./
# With production Form installation depends on 
RUN npm install
# RUN npm run build
# Port number 
EXPOSE 5000
# Start the service 
CMD npm start
 Copy code 

Add one  .dockerignore  file , To exclude files from the container image :

.git
dist
node_modules
npm-debug.log
 Copy code 

At this time, according to the document , We can go through docker Local build image file upload .

that , How can we deploy a project directly from the command line ?

adopt cloudbaserc.json Configure cloud hosting

cloudbase Aspects provide @cloudbase/framework-plugin-container The plug-in allows us to upload files hosted by the cloud , therefore , according to 「cloudbaserc.json file 」 We can do the following configuration :

{
"version": "2.0",
// Environmental Science ID
"envId": "{{env.TCB_ENV_ID}}",
"$schema": "https://framework-1258016615.tcloudbaseapp.com/schema/latest.json",
"framework": {
"name": " name ",
"plugins": {
"service": {
"use": "@cloudbase/framework-plugin-container",
"inputs": {
"serviceName": "serviceName",
"servicePath": "/serviceName-container",
"localPath": ".",
"cpu": 0.25,
"mem": 0.5,
// Port number 
"containerPort": 5000,
"bumpVersion": true,
"dockerfilePath": "./Dockerfile",
"buildDir": "./",
"uploadType": "package",
// environment variable 
"envVariables": {
"NODE_ENV": "production",
"TCB_ENV_ID": "{{env.TCB_ENV_ID}}",
"SECRETID": "{{env.SECRETID}}",
"SECRETKEY": "{{env.SECRETKEY}}",
"BAI_DU_APPID": "{{env.BAI_DU_APPID}}",
"BAI_DU_APP_KEY": "{{env.BAI_DU_APP_KEY}}",
"BAI_DU_APP_SECRET_KEY": "{{env.BAI_DU_APP_SECRET_KEY}}",
"COS_SECRET_ID": "{{env.COS_SECRET_ID}}",
"COS_SECRET_KEY": "{{env.COS_SECRET_KEY}}"
}
}
}
}
}
}
 Copy code 

That's it ?

When building and uploading using cloud hosting , We find that the deployment will always fail , Query the build log through the console , We learned that there was an error in the startup command , therefore , We reconfigured npm start command :

image.png

Added HOSTING environment variable , Used to determine whether it is cloud hosting , In the project file , We modified the previous way of judging cloud functions and local development :

The original way :

// Compatible with cloud functions and local development 
if (process.env.NODE_ENV === 'development') {
await app.listen(port);
} else {
await app.init();
}
// Start development in development mode 
if (process.env.NODE_ENV === 'development') {
bootstrap().then(() => {
console.log(`App listen on http://localhost:${port}`);
});
}
 Copy code 

The way it is now :

// Compatible with cloud functions and local development 
if (isRunInServerModel()) {
await app.listen(port);
} else {
await app.init();
}
 Copy code 

isRunInServerModel() function :

// Run in server mode , That is, it operates by listening to the port 
export const isRunInServerModel = () =>
process.env.NODE_ENV === 'development' ||
process.env.HOSTING;
 Copy code 

Configuration complete , We re uploaded .

It's a success

Hey hey , Upload successful !, meanwhile , Query console , All the environment variables we configured before were also successful , Then I have to try my interface at this time !!

image.png

however , We show up occasionally The internal server reported an error , But we can't find the specific wrong line number 、 Content , This is something we can't stand , therefore , We need to configure our project perfectly !!!

Project configuration

After initializing the build project , We got a simple version of nestjs+cloudbase Of demo service .

code.png

wait !!! It's over ??? So simple ?

If we just run one demo, Try cloud development , that , It's that simple !

However, during business development, we will certainly make more project configurations to meet our business needs , For example, we will filter HTTP abnormal 、 Parameter checking 、 Print logs and so on .

therefore , We need more project configurations for the project .

request body The limitation of

nestjs The bottom layer still uses express, and nodejs Yes body The limit of this is 100kb, If we pass the reference to base64 Pictures of the , It is very likely that parameter transfer fails

{
code: 500,
msg: "Service Error: PayloadTooLargeError: request entity too large"
}
 Copy code 

therefore , We need to configure... In the code body size :

What is commonly used is bodyParser.json(), But this time typescript Will tell us :bodyParser It has been deprecated .

image.png

therefore , We use express.json() Configuration in the form of :

image.png

Parameter verification

nestjs Provides ValidationPipe For us to check the parameters .

 // Parameter checking 
app.useGlobalPipes(new ValidationPipe());
 Copy code 

First : Let's define a validation.pipe.ts file

import { ArgumentMetadata, Injectable, PipeTransform, BadRequestException } from '@nestjs/common';
import { validate } from 'class-validator';
import { plainToClass } from 'class-transformer';
@Injectable()
export class ValidationPipe implements PipeTransform {
// transform(value: any, metadata: ArgumentMetadata) {
// return value;
// }
async transform(value: any, { metatype }: ArgumentMetadata) {
console.log(`value:`, value, 'metatype: ', metatype);
if (!metatype || !this.toValidate(metatype)) {
// If there is no incoming validation rule , We don't verify , Direct return data 
return value;
}
// Convert the object to Class To verify 
const object = plainToClass(metatype, value);
const errors = await validate(object);
if (errors.length > 0) {
const msg = Object.values(errors[0].constraints)[0]; // Just take the first error message and return it 
// Logger.error(`Validation failed: ${msg}`);
throw new BadRequestException(`Validation failed: ${msg}`);
}
return value;
}
private toValidate(metatype: any): boolean {
const types: any[] = [String, Boolean, Number, Array, Object];
return !types.includes(metatype);
}
}
 Copy code 

meanwhile , according to xxx.dto.ts file , We can set the input parameter form

/* * @Author: Lupoy * @Date: 2021-08-31 11:21:17 * @LastEditTime: 2021-09-08 11:06:55 * @LastEditors: Lupoy * @Description: DTO file * @FilePath: src/modules/xxx/xxx.dto.ts */
import { IsNotEmpty, IsString } from 'class-validator';
import DefaultData from '@/modules/baidu/baidu.i18n';
export class xxxDTO {
@IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY })
@IsString({ message: DefaultData.IS_STRING })
readonly base64: string;
}
export class xxxDTO {
@IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY_IN_KOU_TU })
readonly base64: string;
@IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY_IN_KOU_TU })
readonly beautify: number;
@IsNotEmpty({ message: DefaultData.IS_NOT_EMPTY_IN_KOU_TU })
readonly builder: number;
}
 Copy code 

At this time, it can be in controller Use in :

/* * @Author: elizhai * @Date: 2021-08-26 14:51:33 * @LastEditTime: 2021-09-08 11:08:18 */
import { Body, Controller, Post } from '@nestjs/common';
import { xxxService } from '@/modules/xxx/xxx.service';
import { xxxDTO, xxxDTO } from '@/modules/xxx/baidu.dto';
@Controller('x')
export class xxxController {
constructor(private readonly xxxService: xxxService) {}
@Post('bg/xxx')
removeBg(@Body() body: xxxDTO) {
return this.xxxService.removeBg(body.base64);
}
// Body analysis 
@Post('xxx')
xxx(@Body() body: xxxDTO) {
const { base64, beautify, builder } = body;
return this.xxxService.koutu({ data: base64, beautify, builder });
}
}
 Copy code 

When the passed parameters do not meet the specifications we define , It will directly intercept and report an error .

Build a log system

Here we suggest learning 「 Brad tepee 」 Of 「Nest.js From zero to one series ( Four ): Using middleware 、 Interceptor 、 Filters create a log system

Other configuration

nestjs It provides a lot of API For us to set up , for example :app.setGlobalPrefix(' Global routing prefix ')app.disable('x-powered-by') Etc., etc. ......

These can be customized according to the project requirements .

How to freely switch cloud functions and cloud hosting modes ?

How to freely switch between cloud function and cloud hosting upload modes ?

image.png

We define... In the root directory scripts Folder , The folder contains cloud functions and cloud hosting json file , Then write to the root directory by running the script command cloudbaserc.json In the folder .

image.png

/* * @Author: elizhai * @Date: 2021-09-08 10:42:50 * @LastEditTime: 2021-09-08 10:57:54 * @LastEditors: Please set LastEditors * @Description: node Script , Perform cloud function upload or cloud hosting upload * @FilePath: /scripts/index.js */
const fs = require('fs');
const path = require('path');
const fileName =
process.env.FILE_ENV && process.env.FILE_ENV === 'hosting'
? 'cloudbaserc'
: 'cloudbaserc-fx';
fs.readFile(path.join(__dirname, `./${fileName}.json`), 'utf8', function( err, data, ) {
if (err) throw err;
fs.writeFile(path.join(__dirname, '../cloudbaserc.json'), data, 'utf8', err => {
if (err) throw err;
console.log('done');
});
});
 Copy code 

Conclusion

Here we are , We successfully deployed the service , And configured its own project , Give based on cloudbase+ Cloud functions or Cloud hosting + nestjs The basic project is completed .

In the past, due to the need to configure the server , There is a natural gap between the front end and the back end , and cloudbase The cloud functions and cloud hosting provided allow the front end to do back-end projects in some ways , Can make a front-end quickly become a “ Pseudo full stack ”.

All rivers run into sea , Not because the sea is big , Because of the low attitude of the sea

Please bring the original link to reprint ,thank
Similar articles

2021-09-15

2021-09-15

2021-09-15

2021-09-15