Ast, it smells good

Byte data platform front end team 2021-09-15 09:08:49

Bean skin powder , I meet you again , Today's issue , Taro sauce of byte beating data platform , Take everyone in AST The world of .

author : Taro sauce

What is? AST

Abstract syntax tree (Abstract Syntax Tree, AST), Is the tree representation of the abstract syntax structure of the source code , Corresponding to this is the concrete syntax tree ; The reason is abstract , Because the abstract syntax tree does not represent every detail in the real Syntax , And it has nothing to do with grammar 、 It doesn't depend on language details ; You can put AST Imagine a standardized set of programming language interface definitions , It's just this set of specifications , It's for the programming language itself , As small as variable declaration , Large to complex modules , Can be described by this set of specifications , Those who are interested can learn more about it AST The concept and principle of , This paper focuses on JavaScript AST Application .

Why talk about AST

For front-end students , Daily development , and AST Relevant scenes are everywhere ; such as :webpack、babel、 Various lint、prettier、codemod etc. , It's all based on AST To deal with the ; Mastered AST, It is equivalent to mastering the code ability to control the code , It can help us broaden our thinking and vision , Whether it's writing frames , Or write tools and logic ,AST Will be your right-hand man .

AST Analytical process

Recommend one first AST Online conversion website : astexplorer.net , Collect it , Very important ; except js, There are many other languages AST library ; No configuration , As a playground;

In the interpretation of the case Before , First understand the parsing process , There are three steps :

  1. source code --> ast ( The source code is parsed as ast)

  2. traverse ast ( Traverse ast, Access each node in the tree , Do various operations on nodes )

  3. ast --> code ( hold ast Convert to source code , Finish off work )

Source code parsing becomes AST There are many engines , converted AST Be the same in essentials while differing in minor points ;

Use Cases

Start with a variable declaration , as follows :


const dpf = 'DouPiFan';
 Copy code 

Copy the code to astexplorer in , The results are as follows ( The results have been simplified ), This figure explains from the source code to AST The process of ;

Select different third-party libraries to generate AST, The results will vary , Here we use babel/parse For example ; Front end students are right babel I can't be more familiar with , Through its treatment , It can be supported in the browser ES2015+ Code for , This is just babel One of the application scenarios , The official position is :Babel is a javascript compiler.

go back to babel-parser, It USES Babylon As a parsing engine , It is AST To AST The operation of ,babel stay Babylon On the basis of , Encapsulates the parsing (babel-parser) And generation (babel-generator) These two steps , Because you do these two steps every time ; For applications , The point of operation is AST Node Traverse and update 了 ;

first babel plug-in unit

We take one of the simplest babel Plug in as an example , To understand its processing ;

When we develop babel-plugin When , All we need to do is visitor Describe how to AST The conversion of . Add it to your babel Plug in list , You can work , Our first one babel Plug in development is complete ;

babel-plugin-import How is it realized ?

Have used antd Classmate , We all know babel-plugin-import plug-in unit , It's used to do antd On demand loading of components , The effect after configuration is as follows :


import { Button } from 'antd'
↓ ↓ ↓ ↓ ↓ ↓
import Button from 'antd/lib/button'
 Copy code 

The purpose of this paper is , For the implementation details of the plug-in and various boundary conditions , Refer to the plug-in Source code ;

With AST Think with your mind , The implementation steps are as follows :

  1. Find... In the code import sentence , And it must be import { xxx } from 'antd'

  2. Put the node found in step 1 , Convert to import Button from 'antd/lib/button'

Implementation steps

  1. Open artifact : AST Explorer, Copy the first line of code into the artifact

  2. Click... In the code import keyword , It will automatically locate the corresponding node , The structure is as follows :


ImportDeclaration {
type: "ImportDeclaration",
specifiers: [{ // Corresponding {} Components in parentheses
ImportSpecifier: {
type: "ImportSpecifier",
imported: {
type: "Identifier",
name: "Button"
}
}
}]
source: {
type: "StringLiteral",
value: "antd"
},
...
}
 Copy code 

The source code is converted into objects with types and attributes , Whether it's a keyword 、 Variable declarations , Or literal value , There are corresponding types ;

  1. import The corresponding type of statement is : ImportDeclaration

  2. { Button } The corresponding is specifiers Array , Only... Is introduced in the example "Button", therefore specifiers There is only one element in the array

  3. specifiers The elements in , That is to say Button, The type is ImportSpecifier;

  4. 'antd' stay source In nodes , The type is :StringLiteral,value by antd

Explain again : The example is not a complete logical implementation , Details and boundary conditions , You can refer to the source code or improve it yourself ;

in the light of AST The operation of , And the browser comes with DOM API similar ; First determine the type of node to find , Then according to the specific conditions , Narrow your search , Finally, for the found node , Add, delete, modify, etc ;


// babel Plug in templates
export default function({types: t}) {
return {
// Visitor Each function in receives 2 Parameters :path and state
visitor: {
ImportDeclaration(path, state) {
const { node } = path;
// source The value of is antd
if(node.source.value === 'antd'){
const specifiers = node.specifiers
// Traverse specifiers Array
const result = specifiers.map((specifier) => {
const local = specifier.local
// structure source
const source = t.stringLiteral(`${node.source.value}/lib/${local.name}`)
// structure import sentence
return t.importDeclaration([t.importDefaultSpecifier(local)], source)
})
console.log(result)
path.replaceWithMultiple(result)
}
}
}
}
}
 Copy code 

The verification method is also very simple , Copy this code to AST Explorer in , Just view the output results ; Come here , This “ Simple and easy ” Plug in implementation complete ;

Let's review the implementation ideas :

  1. Compare the differences of the source code in the syntax tree , Specify what transformations and modifications to make

  2. Analysis type , Can be in babel official , Find type description

  3. In the plug-in template , adopt visitor Access the corresponding type node , Add, delete, modify, etc

Codemod

It explains ast stay babel Basic operation methods in , Look again. codemod.

Use antd3 Classmate , I've been in contact with them all antd3 To antd4 Of codemod, This is a tool that helps us automate , hold antd3 To antd4 A tool library for ; Because its essence is code conversion , So based on the babel Realization codemod, It's completely ok Of . But apart from transcoding , You also need a command line operation , Source code reading , Batch execution conversion , Log output and other functions , It is a collection of functions , Transcoding is a very important part of it ; therefore , Recommend another tool jscodeshift. His position is a transform runner, therefore , Our core work is , Define a series of transform, That is, conversion rules , The rest of the command line 、 Source code reading 、 Batch execution 、 Log output can be given to jscodeshift.

preparation

So let's define one transform, and babel The plug-in looks like


import { Transform } from "jscodeshift";
const transform: Transform = (file, api, options) => {
return null;
};
export default transform;
 Copy code 
Hands-on practice

We tried to Button Component's "type" Property is replaced with "status", And put width attribute , Add to style in :


// Input
const Component = () => {
return (
<Button
type="dange"
width="20"
/>
)
}
// Output
const Component = () => {
return (
<Button
statue="dange"
style={{
width: 20
}}
/>
)
}
 Copy code 
Differences in contrast
  1. react The attribute type of the component is :JSXIdentifier, attribute "type" It is amended as follows "status"

  2. If the component has "width" attribute , Move the attribute to "style" Properties of the

lookup Button The code for the component is as follows :


import { Transform } from "jscodeshift";
const transform = (file, api, options) => {
const j = api.jscodeshift;
// lookup jsx node , adopt find The second parameter of the method
return j(file.source).find(j.JSXOpeningElement, {
name: {
type: 'JSXIdentifier',
name: 'Button'
}
})
};
export default transform;
 Copy code 
Property substitution

Next , Add attribute replacement logic , hold type Replace with status


export default function transformer(file, api) {
const j = api.jscodeshift;
return j(file.source)
.find(j.JSXOpeningElement, {
name: {
type: 'JSXIdentifier',
name: 'Button'
}
}).forEach(function(path){
var attributes = path.value.attributes;
attributes.forEach(function(node, index){
const attr = node.name.name;
if(attr === 'type'){
// attr by type when , Replace the property name with status
node.name.name = 'status'
}
})
})
.toSource();
}
 Copy code 

Looking for JSX Element time ,jscodeshift You can get it directly :j(file.source).findJSXElements() , Use here find Instead of ,find Second parameter of , Filter conditions can be described ;

jscodeshift Support chain calls , After finding the node , Use forEach Traverse , When the component's property name is type when , Replace the property name with "status", Only one case is considered here , There is still JSXNamespaceName Scene , such as : ;

Handle width

There is width when , obtain width Value , Then delete the node ;

The next step is to create style node , The type is jsxAttribute, hold width The value of is set back to style


...
attributes.forEach(function(node, index){
const attr = node.name.name;
if(attr === 'width'){
// obtain width Value
width = node.value.value;
// Delete width attribute
attributes.splice(index, 1)
}
let width;
if(width){
// structure style node
var node = j.jsxAttribute(
// Set up attr For the name of the : style
j.jsxIdentifier('style'),
// structure jsxExpressionContainer { }
// structure objectExpression
j.jsxExpressionContainer(j.objectExpression([
j.objectProperty(
j.identifier('width'),
j.stringLiteral(width),
),
])),
)
// Insert style node
attributes.splice(index, 0, node)
}
}
...
 Copy code 

summary

The above describes the methods based on babel Implementation and jscodeshift The implementation of the , The same way of thinking , Relatively simple , But it takes extra time and energy to reach a perfect state , Especially in the face of large-scale code processing , There are many boundary conditions , What needs to be considered is very comprehensive ; But the investment is worth it , Can automate most of the work ;

in addition ,babel My specialty is in ast To deal with ,jscodeshift More like a fully functional set of tools , You can focus on the implementation of the converter , Please select the appropriate tool according to the actual scene .

Byte beating data platform front-end team , Responsible for the research and development of big data related products in the company . We maintain a strong enthusiasm for front-end technology , In addition to the research and development related to data products , In data visualization 、 Massive data processing optimization 、web excel、WebIDE、 Privatization deployment 、 There is a lot of exploration and accumulation in engineering tools , If you are interested, you can contact us .

Please bring the original link to reprint ,thank
Similar articles

2021-09-15

2021-09-15