phoedos/pmd
1
1
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
pmd/pmd_devdocs_major_adding_new_language_antlr.html

2283 lines
69 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="How to add a new language to PMD using ANTLR grammar.">
<meta name="keywords" content="devdocsextending, ">
<title>Adding PMD support for a new ANTLR grammar based language | PMD Source Code Analyzer</title>
<link rel="stylesheet" type="text/css" href="assets/fontawesome-free-5.15.4-web/css/all.min.css">
<link rel="stylesheet" type="text/css" href="assets/bootstrap-4.5.2-dist/css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="css/syntax.css">
<link rel="stylesheet" type="text/css" href="css/modern-business.css">
<link rel="stylesheet" type="text/css" href="css/customstyles.css">
<link rel="stylesheet" type="text/css" href="css/theme-green.css">
<link rel="stylesheet" type="text/css" href="css/pmd-customstyles.css">
<link rel="shortcut icon" href="images/logo/favicon.ico" type="image/x-icon">
<link rel="icon" href="images/logo/favicon.ico" type="image/x-icon">
<link rel="alternate" type="application/rss+xml" title="" href="feed.xml">
</head>
<body>
<!-- Content is offset by the height of the topnav bar. -->
<!-- There's already a padding-top rule in modern-business.css, but it apparently doesn't work on Firefox 60 and Chrome 67 -->
<div id="topbar-content-offset">
<!-- Navigation -->
<nav class="navbar navbar-expand-lg fixed-top navbar-dark">
<div class="container topnavlinks">
<a class="navbar-brand fas fa-home fa-lg" href="index.html">&nbsp;<span class="projectTitle"> PMD Source Code Analyzer Project</span></a>
<button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarSupportedContent">
<ul class="navbar-nav mr-auto mt-2 mt-lg-0"></ul>
<ul class="navbar-nav">
<!-- toggle sidebar button -->
<li class="nav-item"><a id="tg-sb-link" class="nav-link" href="#"><i id="tg-sb-icon" class="fas fa-toggle-on"></i> Nav</a></li>
<!-- entries without drop-downs appear here -->
<li class="nav-item"><a class="nav-link" href="https://github.com/pmd/pmd/releases/latest" target="_blank">Download</a></li>
<li class="nav-item"><a class="nav-link" href="https://github.com/pmd/pmd" target="_blank">Fork us on github</a></li>
<!-- entries with drop-downs appear here -->
<!-- conditional logic to control which topnav appears for the audience defined in the configuration file.-->
</ul>
<form class="form-inline my-2 my-lg-0">
<input class="form-control mr-sm-2" type="search" placeholder="search..." id="search-input">
<ul id="results-container"></ul>
</form>
</div>
</div>
</nav>
<!-- Page Content -->
<div class="container-toc-wrapper">
<div class="container">
<div class="col-lg-12">&nbsp;</div>
<!-- Content Row -->
<div class="row">
<!-- Sidebar Column -->
<div class="col-md-3" id="tg-sb-sidebar">
<ul id="mysidebar" class="nav">
<li class="sidebarTitle">PMD 7.8.0-SNAPSHOT</li>
<div class="sidebarTitleDate">Release date: 29-November-2024</div>
<li>
<a href="#">About</a>
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="pmd_release_notes.html">Release notes</a></li>
<li><a href="pmd_release_notes_pmd7.html">Release notes (PMD 7)</a></li>
<li><a href="pmd_about_help.html">Getting help</a></li>
<li><a href="pmd_about_release_policies.html">Release policies</a></li>
<li><a href="pmd_about_support_lifecycle.html">Support lifecycle</a></li>
</ul>
</li>
<li>
<a href="#">User Documentation</a>
<ul>
<li><a href="pmd_userdocs_migrating_to_pmd7.html">Migration Guide for PMD 7</a></li>
<li><a href="pmd_userdocs_installation.html">Installation and basic CLI usage</a></li>
<li><a href="pmd_userdocs_making_rulesets.html">Making rulesets</a></li>
<li><a href="pmd_userdocs_configuring_rules.html">Configuring rules</a></li>
<li><a href="pmd_userdocs_best_practices.html">Best practices</a></li>
<li><a href="pmd_userdocs_suppressing_warnings.html">Suppressing warnings</a></li>
<li><a href="pmd_userdocs_incremental_analysis.html">Incremental analysis</a></li>
<li><a href="pmd_userdocs_cli_reference.html">PMD CLI reference</a></li>
<li><a href="pmd_userdocs_report_formats.html">PMD Report formats</a></li>
<li><a href="pmd_userdocs_3rdpartyrulesets.html">3rd party rulesets</a></li>
<li class="subfolders">
<a href="#">CPD reference</a>
<ul>
<li><a href="pmd_userdocs_cpd.html">Copy-paste detection</a></li>
<li><a href="pmd_userdocs_cpd_report_formats.html">CPD Report formats</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Extending PMD</a>
<ul>
<li><a href="pmd_userdocs_extending_writing_rules_intro.html">Introduction to writing rules</a></li>
<li><a href="pmd_userdocs_extending_your_first_rule.html">Your first rule</a></li>
<li><a href="pmd_userdocs_extending_writing_xpath_rules.html">XPath rules</a></li>
<li><a href="pmd_userdocs_extending_writing_java_rules.html">Java rules</a></li>
<li><a href="pmd_userdocs_extending_designer_reference.html">Rule designer reference</a></li>
<li><a href="pmd_userdocs_extending_defining_properties.html">Defining rule properties</a></li>
<li><a href="pmd_userdocs_extending_rule_guidelines.html">Rule guidelines</a></li>
<li><a href="pmd_userdocs_extending_testing.html">Testing your rules</a></li>
<li><a href="pmd_userdocs_extending_ast_dump.html">Creating (XML) dump of the AST</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Tools / Integrations</a>
<ul>
<li><a href="pmd_userdocs_tools_maven.html">Maven PMD Plugin</a></li>
<li><a href="pmd_userdocs_tools_gradle.html">Gradle</a></li>
<li><a href="pmd_userdocs_tools_ant.html">Ant</a></li>
<li><a href="pmd_userdocs_tools_java_api.html">PMD Java API</a></li>
<li><a href="pmd_userdocs_tools_bld.html">bld PMD Extension</a></li>
<li><a href="pmd_userdocs_tools_ci.html">CI integrations</a></li>
<li><a href="pmd_userdocs_tools.html">Other Tools / Integrations</a></li>
</ul>
</li>
</ul>
</li>
<li>
<a href="#">Rule Reference</a>
<ul>
<li class="subfolders">
<a href="#">Apex Rules</a>
<ul>
<li><a href="pmd_rules_apex.html">Index</a></li>
<li><a href="pmd_rules_apex_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_apex_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_apex_design.html">Design</a></li>
<li><a href="pmd_rules_apex_documentation.html">Documentation</a></li>
<li><a href="pmd_rules_apex_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_apex_performance.html">Performance</a></li>
<li><a href="pmd_rules_apex_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">HTML Rules</a>
<ul>
<li><a href="pmd_rules_html.html">Index</a></li>
<li><a href="pmd_rules_html_bestpractices.html">Best Practices</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Java Rules</a>
<ul>
<li><a href="pmd_rules_java.html">Index</a></li>
<li><a href="pmd_rules_java_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_java_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_java_design.html">Design</a></li>
<li><a href="pmd_rules_java_documentation.html">Documentation</a></li>
<li><a href="pmd_rules_java_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_java_multithreading.html">Multithreading</a></li>
<li><a href="pmd_rules_java_performance.html">Performance</a></li>
<li><a href="pmd_rules_java_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Java Server Pages Rules</a>
<ul>
<li><a href="pmd_rules_jsp.html">Index</a></li>
<li><a href="pmd_rules_jsp_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_jsp_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_jsp_design.html">Design</a></li>
<li><a href="pmd_rules_jsp_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_jsp_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">JavaScript Rules</a>
<ul>
<li><a href="pmd_rules_ecmascript.html">Index</a></li>
<li><a href="pmd_rules_ecmascript_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_ecmascript_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_ecmascript_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_ecmascript_performance.html">Performance</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Kotlin Rules</a>
<ul>
<li><a href="pmd_rules_kotlin.html">Index</a></li>
<li><a href="pmd_rules_kotlin_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_kotlin_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Maven POM Rules</a>
<ul>
<li><a href="pmd_rules_pom.html">Index</a></li>
<li><a href="pmd_rules_pom_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Modelica Rules</a>
<ul>
<li><a href="pmd_rules_modelica.html">Index</a></li>
<li><a href="pmd_rules_modelica_bestpractices.html">Best Practices</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">PLSQL Rules</a>
<ul>
<li><a href="pmd_rules_plsql.html">Index</a></li>
<li><a href="pmd_rules_plsql_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_plsql_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_plsql_design.html">Design</a></li>
<li><a href="pmd_rules_plsql_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Salesforce Visualforce Rules</a>
<ul>
<li><a href="pmd_rules_visualforce.html">Index</a></li>
<li><a href="pmd_rules_visualforce_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Scala Rules</a>
<ul>
<li><a href="pmd_rules_scala.html">Index</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Swift Rules</a>
<ul>
<li><a href="pmd_rules_swift.html">Index</a></li>
<li><a href="pmd_rules_swift_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_swift_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Velocity Template Language (VTL) Rules</a>
<ul>
<li><a href="pmd_rules_velocity.html">Index</a></li>
<li><a href="pmd_rules_velocity_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_velocity_design.html">Design</a></li>
<li><a href="pmd_rules_velocity_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">WSDL Rules</a>
<ul>
<li><a href="pmd_rules_wsdl.html">Index</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">XML Rules</a>
<ul>
<li><a href="pmd_rules_xml.html">Index</a></li>
<li><a href="pmd_rules_xml_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_xml_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">XSL Rules</a>
<ul>
<li><a href="pmd_rules_xsl.html">Index</a></li>
<li><a href="pmd_rules_xsl_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_xsl_performance.html">Performance</a></li>
</ul>
</li>
</ul>
</li>
<li>
<a href="#">Language-Specific Documentation</a>
<ul>
<li><a href="pmd_languages_index.html">Overview</a></li>
<li><a href="pmd_languages_configuration.html">Language configuration</a></li>
<li><a href="pmd_languages_apex.html">Apex</a></li>
<li><a href="pmd_languages_cpp.html">C/C++</a></li>
<li><a href="pmd_languages_cs.html">C#</a></li>
<li><a href="pmd_languages_coco.html">Coco</a></li>
<li><a href="pmd_languages_dart.html">Dart</a></li>
<li><a href="pmd_languages_fortran.html">Fortran</a></li>
<li><a href="pmd_languages_gherkin.html">Gherkin</a></li>
<li><a href="pmd_languages_go.html">Go</a></li>
<li><a href="pmd_languages_html.html">HTML</a></li>
<li><a href="pmd_languages_java.html">Java</a></li>
<li><a href="pmd_languages_js_ts.html">JavaScript / TypeScript</a></li>
<li><a href="pmd_languages_jsp.html">JSP</a></li>
<li><a href="pmd_languages_julia.html">Julia</a></li>
<li><a href="pmd_languages_kotlin.html">Kotlin</a></li>
<li><a href="pmd_languages_lua.html">Lua</a></li>
<li><a href="pmd_languages_matlab.html">Matlab</a></li>
<li><a href="pmd_languages_modelica.html">Modelica</a></li>
<li><a href="pmd_languages_objectivec.html">Objective-C</a></li>
<li><a href="pmd_languages_perl.html">Perl</a></li>
<li><a href="pmd_languages_php.html">PHP</a></li>
<li><a href="pmd_languages_plsql.html">PLSQL</a></li>
<li><a href="pmd_languages_python.html">Python</a></li>
<li><a href="pmd_languages_ruby.html">Ruby</a></li>
<li><a href="pmd_languages_scala.html">Scala</a></li>
<li><a href="pmd_languages_swift.html">Swift</a></li>
<li><a href="pmd_languages_tsql.html">T-SQL</a></li>
<li><a href="pmd_languages_visualforce.html">Visualforce</a></li>
<li><a href="pmd_languages_velocity.html">Velocity Template Language (VTL)</a></li>
<li><a href="pmd_languages_xml.html">XML and XML dialects</a></li>
</ul>
</li>
<li>
<a href="#">Developer Documentation</a>
<ul>
<li><a href="pmd_devdocs_development.html">Developer resources</a></li>
<li><a href="pmd_devdocs_building.html">Building PMD from source</a></li>
<li><a href="https://github.com/pmd/pmd/blob/main/CONTRIBUTING.md" target="_blank">Contributing</a></li>
<li><a href="pmd_devdocs_writing_documentation.html">Writing documentation</a></li>
<li><a href="pmd_devdocs_roadmap.html">Roadmap</a></li>
<li><a href="pmd_devdocs_how_pmd_works.html">How PMD works</a></li>
<li><a href="pmd_devdocs_pmdtester.html">Pmdtester</a></li>
<li><a href="pmd_devdocs_rule_deprecation_policy.html">Rule Deprecation Policy</a></li>
<li class="subfolders">
<a href="#">Major contributions</a>
<ul>
<li><a href="pmd_devdocs_major_rule_guidelines.html">Rule Guidelines</a></li>
<li><a href="pmd_devdocs_major_adding_new_language_javacc.html">Adding a new language (JavaCC)</a></li>
<li class="active"><a href="pmd_devdocs_major_adding_new_language_antlr.html">Adding a new language (ANTLR)</a></li>
<li><a href="pmd_devdocs_major_adding_new_cpd_language.html">Adding a new CPD language</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Experimental features</a>
<ul>
<li><a href="tag_experimental.html">List of experimental Features</a></li>
</ul>
</li>
</ul>
</li>
<li>
<a href="#">Project documentation</a>
<ul>
<li class="subfolders">
<a href="#">Trivia about PMD</a>
<ul>
<li><a href="pmd_projectdocs_trivia_news.html">PMD in the press</a></li>
<li><a href="pmd_projectdocs_trivia_products.html">Products & books related to PMD</a></li>
<li><a href="pmd_projectdocs_trivia_similarprojects.html">Similar projects</a></li>
<li><a href="pmd_projectdocs_trivia_meaning.html">What does 'PMD' mean?</a></li>
</ul>
</li>
<li><a href="pmd_projectdocs_logo.html">Logo</a></li>
<li><a href="pmd_projectdocs_faq.html">FAQ</a></li>
<li><a href="license.html">License</a></li>
<li><a href="pmd_projectdocs_credits.html">Credits</a></li>
<li><a href="pmd_release_notes_old.html">Old release notes</a></li>
<li><a href="pmd_projectdocs_decisions.html">Decisions</a></li>
<li class="subfolders">
<a href="#">Project management</a>
<ul>
<li><a href="pmd_projectdocs_committers_infrastructure.html">Infrastructure</a></li>
<li><a href="pmd_projectdocs_committers_releasing.html">Release process</a></li>
<li><a href="pmd_projectdocs_committers_merging_pull_requests.html">Merging pull requests</a></li>
<li><a href="pmd_projectdocs_committers_main_landing_page.html">Main Landing page</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<!-- Content Column -->
<div class="col-md-9" id="tg-sb-content">
<header>
<div class="row">
<div class="col-lg-12">
<a href="./" role="button"
><i class="fa fa-home fa-lg"></i
></a>
» Adding PMD support for a new ANTLR grammar based language
<a
target="_blank"
href="https://github.com/pmd/pmd/blob/main/docs/pages/pmd/devdocs/major_contributions/adding_a_new_antlr_based_language.md"
class="float-right"
role="button"
><i class="fab fa-github fa-lg"></i> Edit on GitHub</a
>
</div>
</div>
<hr />
</header>
<div class="post-header">
<h1 class="post-title-main">Adding PMD support for a new ANTLR grammar based language</h1>
</div>
<div class="post-content" data-github-edit-url="https://github.com/pmd/pmd/blob/main/docs/pages/pmd/devdocs/major_contributions/adding_a_new_antlr_based_language.md">
<div class="summary">How to add a new language to PMD using ANTLR grammar.</div>
<details id="inline-toc-details">
<summary>Table of Contents</summary>
<div id="inline-toc"><!-- empty, move TOC here when screen size too small --></div>
</details>
<div class="bs-callout bs-callout-warning">
<strong>Before you start…</strong><br /><br />
This is really a big contribution and can’t be done with a drive by contribution. It requires dedicated passion
and long commitment to implement support for a new language.<br /><br />
This step-by-step guide is just a small intro to get the basics started, and it’s also not necessarily up-to-date
or complete. You have to be able to fill in the blanks.<br /><br />
Currently, the Antlr integration has some basic <strong>limitations</strong> compared to JavaCC: The output of the
Antlr parser generator is not an abstract syntax tree (AST) but a parse tree (also known as CST, concrete syntax tree).
As such, a parse tree is much more fine-grained than what a typical JavaCC grammar will produce. This means that the
parse tree is much deeper and contains nodes down to the different token types.<br /><br />
The Antlr nodes are context objects and serve a different abstraction than nodes in an AST. These context objects
themselves don’t have any attributes because they themselves represent the attributes (as nodes or leaves in the
parse tree). As they don’t have attributes, there are no attributes that can be used in XPath based rules.<br /><br />
The current implementation of the languages using ANTLR use these context objects as nodes in PMD’s AST
representation.<br /><br />
In order to overcome these limitations, one would need to implement a post-processing step that transforms
a parse tree into an abstract syntax tree and introducing real nodes on a higher abstraction level. These
real nodes can then have attributes which are available in XPath based rules. The transformation can happen
with a visitor, but the implementation of the AST is a manual step. This step is <strong>not</strong> described
in this guide.<br /><br />
After the basic support for a language is there, there are lots of missing features left. Typical features
that can greatly improve rule writing are: symbol table, type resolution, call/data flow analysis.<br /><br />
Symbol table keeps track of variables and their usages. Type resolution tries to find the actual class type
of each used type, following along method calls (including overloaded and overwritten methods), allowing
to query subtypes and type hierarchy. This requires additional configuration of an auxiliary classpath.
Call and data flow analysis keep track of the data as it is moving through different execution paths
a program has.<br /><br />
These features are out of scope of this guide. Type resolution and data flow are features that
definitely don’t come for free. It is much effort and requires perseverance to implement.<br /><br />
</div>
<h2 id="steps">Steps</h2>
<h3 id="1--start-with-a-new-sub-module">1. Start with a new sub-module</h3>
<ul>
<li>See pmd-swift for examples.</li>
<li>Make sure to add your new module to PMD’s parent pom as <code class="language-plaintext highlighter-rouge">&lt;module&gt;</code> entry, so that it is built alongside the
other languages.</li>
<li>Also add your new module to the dependencies list in “pmd-languages-deps/pom.xml”, so that the new language
is automatically available in the binary distribution (pmd-dist).</li>
</ul>
<h3 id="2--implement-an-ast-parser-for-your-language">2. Implement an AST parser for your language</h3>
<ul>
<li>ANTLR will generate the parser for you based on the grammar file. The grammar file needs to be placed in the
folder <code class="language-plaintext highlighter-rouge">src/main/antlr4</code> in the appropriate sub package <code class="language-plaintext highlighter-rouge">ast</code> of the language. E.g. for swift, the grammar
file is <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/antlr4/net/sourceforge/pmd/lang/swift/ast/Swift.g4">Swift.g4</a>
and is placed in the package <code class="language-plaintext highlighter-rouge">net.sourceforge.pmd.lang.swift.ast</code>.</li>
<li>Configure the options “superClass” and “contextSuperClass”. These are the base classes for the generated
classes.</li>
</ul>
<h3 id="3--create-ast-node-classes">3. Create AST node classes</h3>
<ul>
<li>The individual AST nodes are generated, but you need to define the common interface for them.</li>
<li>You need to define the supertype interface for all nodes of the language. For that, we provide
<a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrNode.java"><code class="language-plaintext highlighter-rouge">AntlrNode</code></a>.</li>
<li>See <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/SwiftNode.java"><code class="language-plaintext highlighter-rouge">SwiftNode</code></a>
as an example.</li>
<li>Additionally, you need several base classes:
<ul>
<li>a language specific inner node - these nodes represent the production rules from the grammar.
In Antlr, they are called “ParserRuleContext”. We call them “InnerNode”. Use the
base class from pmd-core
<a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/BaseAntlrInnerNode.java"><code class="language-plaintext highlighter-rouge">BaseAntlrInnerNode</code></a>
. And example is <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/SwiftInnerNode.java"><code class="language-plaintext highlighter-rouge">SwiftInnerNode</code></a>.
Note that this language specific inner node is package-private, as it is only the base class for the concrete
nodes generated by ANLTR.</li>
<li>a language specific root node - this provides the root of the AST and our parser will return
subtypes of this node. The root node itself is a “InnerNode”.
See <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/SwiftRootNode.java"><code class="language-plaintext highlighter-rouge">SwiftRootNode</code></a>.
Note that this language specific root node is package-private, as it is only the base class for the concrete
node generated by ANLTR.</li>
<li>a language specific terminal node.
See <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/SwiftTerminalNode.java"><code class="language-plaintext highlighter-rouge">SwiftTerminalNode</code></a>.</li>
<li>a language specific error node.
See <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/SwiftErrorNode.java"><code class="language-plaintext highlighter-rouge">SwiftErrorNode</code></a>.</li>
<li>a language name dictionary. This is used to convert ANTLR node names to useful XPath node names.
See <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/SwiftNameDictionary.java">`SwiftNameDictionary’</a>.</li>
</ul>
</li>
<li>Once these base classes exist, you need to change the ANTLR grammar to add additional members via <code class="language-plaintext highlighter-rouge">@parser::members</code>
<ul>
<li>Define a package private field <code class="language-plaintext highlighter-rouge">DICO</code> which creates a new instance of your language name dictionary using the
vocabulary from the generated parser (<code class="language-plaintext highlighter-rouge">VOCABULARY</code>).</li>
<li>Define two additional methods to help converting the ANTLR context objects into PMD AST nodes.
The methods are abstract in <a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrGeneratedParserBase.java"><code class="language-plaintext highlighter-rouge">AntlrGeneratedParserBase</code></a>
and need to be implemented here for the concrete language: <code class="language-plaintext highlighter-rouge">createPmdTerminal()</code> and <code class="language-plaintext highlighter-rouge">createPmdError()</code>.</li>
</ul>
</li>
<li>In order for the generated code to match and use our custom classes, we have a common ant script, that fiddles with
the generated code. The ant script is <a href="https://github.com/pmd/pmd/blob/main/antlr4-wrapper.xml"><code class="language-plaintext highlighter-rouge">antlr4-wrapper.xml</code></a>
and does not need to be adjusted - it has plenty of parameters that can be configured.
The ant script is added in the language module’s <code class="language-plaintext highlighter-rouge">pom.xml</code> where the parameters are set (e.g. name of root name
class). Have a look at Swift’s example: <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/pom.xml"><code class="language-plaintext highlighter-rouge">pmd-swift/pom.xml</code></a>.</li>
<li>You can add additional methods in your “InnerNode” (e.g. <code class="language-plaintext highlighter-rouge">SwiftInnerNode</code>) that are available on all nodes.
But on most cases you won’t need to do anything.</li>
</ul>
<h3 id="4--generate-your-parser-using-antlr">4. Generate your parser (using ANTLR)</h3>
<ul>
<li>Make sure, you have the property <code class="language-plaintext highlighter-rouge">&lt;antlr4.visitor&gt;true&lt;/antlr4.visitor&gt;</code> in your <code class="language-plaintext highlighter-rouge">pom.xml</code> file.</li>
<li>This is just a matter of building the language module. ANTLR is called via ant, and this step is added
to the phase <code class="language-plaintext highlighter-rouge">generate-sources</code>. So you can just call e.g. <code class="language-plaintext highlighter-rouge">./mvnw generate-sources -pl pmd-swift</code> to
have the parser generated.</li>
<li>The generated code will be placed under <code class="language-plaintext highlighter-rouge">target/generated-sources/antlr4</code> and will not be committed to
source control.</li>
<li>You should review <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/pom.xml"><code class="language-plaintext highlighter-rouge">pmd-swift/pom.xml</code></a>.</li>
</ul>
<h3 id="5--create-a-tokenmanager">5. Create a TokenManager</h3>
<ul>
<li>This is needed to support CPD (copy paste detection)</li>
<li>We provide a default implementation using <a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrTokenManager.java"><code class="language-plaintext highlighter-rouge">AntlrTokenManager</code></a>.</li>
<li>You must create your own “AntlrCpdLexer” such as we do with
<a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/cpd/SwiftCpdLexer.java"><code class="language-plaintext highlighter-rouge">SwiftCpdLexer</code></a>.</li>
<li>
<p>If you wish to filter specific tokens (e.g. comments to support CPD suppression via “CPD-OFF” and “CPD-ON”)
you can create your own implementation of
<a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/cpd/impl/AntlrTokenFilter.java"><code class="language-plaintext highlighter-rouge">AntlrTokenFilter</code></a>.
You’ll need to override then the protected method <code class="language-plaintext highlighter-rouge">getTokenFilter(AntlrTokenManager)</code>
and return your custom filter. See the CpdLexer for C# as an exmaple:
<a href="https://github.com/pmd/pmd/blob/main/pmd-cs/src/main/java/net/sourceforge/pmd/lang/cs/cpd/CsCpdLexer.java"><code class="language-plaintext highlighter-rouge">CsCpdLexer</code></a>.</p>
<p>If you don’t need a custom token filter, you don’t need to override the method. It returns the default
<code class="language-plaintext highlighter-rouge">AntlrTokenFilter</code> which doesn’t filter anything.</p>
</li>
</ul>
<h3 id="6--create-a-pmd-parser-adapter">6. Create a PMD parser “adapter”</h3>
<ul>
<li>Create your own parser, that adapts the ANLTR interface to PMD’s parser interface.</li>
<li>We provide a <a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrBaseParser.java"><code class="language-plaintext highlighter-rouge">AntlrBaseParser</code></a>
implementation that you need to extend to create your own adapter as we do with
<a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/PmdSwiftParser.java"><code class="language-plaintext highlighter-rouge">PmdSwiftParser</code></a>.</li>
</ul>
<h3 id="7--create-a-language-version-handler">7. Create a language version handler</h3>
<ul>
<li>Now you need to create your version handler, as we did with <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/SwiftHandler.java"><code class="language-plaintext highlighter-rouge">SwiftHandler</code></a>.</li>
<li>This class is sort of a gateway between PMD and all parsing logic specific to your language.</li>
<li>For a minimal implementation, it just needs to return a parser <em>(see step #6)</em>.</li>
<li>It can be used to provide other features for your language like
<ul>
<li>violation suppression logic</li>
<li><a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/reporting/ViolationDecorator.html#"><code>ViolationDecorator</code></a>s, to add additional language specific information to the
created violations. The <a href="pmd_languages_java.html#violation-decorators">Java language module</a> uses this to
provide the method name or class name, where the violation occurred.</li>
<li>metrics</li>
<li>custom XPath functions</li>
</ul>
</li>
</ul>
<h3 id="8--create-a-base-visitor">8. Create a base visitor</h3>
<ul>
<li>A parser visitor adapter is not needed anymore with PMD 7. The visitor interface now provides a default
implementation.</li>
<li>The visitor for ANTLR based AST is generated along the parser from the ANTLR grammar file. The
base interface for a visitor is <a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/AstVisitor.java"><code class="language-plaintext highlighter-rouge">AstVisitor</code></a>.</li>
<li>The generated visitor class for Swift is called <code class="language-plaintext highlighter-rouge">SwiftVisitor</code>.</li>
<li>In order to help use this visitor later on, a base visitor class should be created.
See <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/ast/SwiftVisitorBase.java"><code class="language-plaintext highlighter-rouge">SwiftVisitorBase</code></a>
as an example.</li>
</ul>
<h3 id="9-make-pmd-recognize-your-language">9. Make PMD recognize your language</h3>
<ul>
<li>Create your own subclass of <code class="language-plaintext highlighter-rouge">net.sourceforge.pmd.lang.impl.SimpleLanguageModuleBase</code>, see Swift as an example:
<a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/SwiftLanguageModule.java"><code class="language-plaintext highlighter-rouge">SwiftLanguageModule</code></a>.</li>
<li>Add for each version of your language a call to <code class="language-plaintext highlighter-rouge">addVersion</code> in your language module’s constructor.
Use <code class="language-plaintext highlighter-rouge">addDefaultVersion</code> for defining the default version.</li>
<li>You’ll need to refer the language version handler created in step #7.</li>
<li>Create the service registration via the text file <code class="language-plaintext highlighter-rouge">src/main/resources/META-INF/services/net.sourceforge.pmd.lang.Language</code>.
Add your fully qualified class name as a single line into it.</li>
</ul>
<h3 id="10-create-an-abstract-rule-class-for-the-language">10. Create an abstract rule class for the language</h3>
<ul>
<li>You need to create your own abstract rule class in order to interface your language with PMD’s generic rule
execution.</li>
<li>See <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/rule/AbstractSwiftRule.java"><code class="language-plaintext highlighter-rouge">AbstractSwiftRule</code></a> as an example.</li>
<li>The rule basically just extends
<a href="https://github.com/pmd/pmd/blob/main/pmd-core/src/main/java/net/sourceforge/pmd/lang/rule/AbstractVisitorRule.java"><code class="language-plaintext highlighter-rouge">AbstractVisitorRule</code></a>
and only redefines the abstract <code class="language-plaintext highlighter-rouge">buildVisitor()</code> method to return our own type of visitor.
In this case our <code class="language-plaintext highlighter-rouge">SwiftVisitor</code> is used.
While there is no real functionality added, every language should have its own base class for rules.
This helps to organize the code.</li>
<li>All other rules for your language should extend this class. The purpose of this class is to provide a visitor
via the method <code class="language-plaintext highlighter-rouge">buildVisitor()</code> for analyzing the AST. The provided visitor only implements the visit methods
for specific AST nodes. The other node types use the default behavior, and you don’t need to care about them.</li>
<li>Note: This is different from how it was in PMD 6: Each rule in PMD 6 was itself a visitor (implementing the visitor
interface of the specific language). Now the rule just provides a visitor, which can be hidden and potentially
shared between rules.</li>
</ul>
<h3 id="11-create-rules">11. Create rules</h3>
<ul>
<li>Creating rules is already pretty well documented in PMD - and it’s no different for a new language, except you
may have different AST nodes.</li>
<li>PMD supports 2 types of rules, through visitors or XPath.</li>
<li>To add a visitor rule:
<ul>
<li>You need to extend the abstract rule you created on the previous step, you can use the swift
rule <a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/rule/bestpractices/UnavailableFunctionRule.java">UnavailableFunctionRule</a>
as an example. Note, that all rule classes should be suffixed with <code class="language-plaintext highlighter-rouge">Rule</code> and should be placed
in a package the corresponds to their category.</li>
</ul>
</li>
<li>To add an XPath rule you can follow our guide <a href="pmd_userdocs_extending_writing_xpath_rules.html">Writing XPath Rules</a>.</li>
<li>When creating the category ruleset XML file, the XML can reference build properties that are replaced
during the build. This is used for the <code class="language-plaintext highlighter-rouge">externalInfoUrl</code> attribute of a rule. E.g. we use <code class="language-plaintext highlighter-rouge">${pmd.website.baseurl}</code>
to point to the correct webpage (depending on the PMD version). In order for this to work, you need to add a
resource filtering configuration in the language module’s <code class="language-plaintext highlighter-rouge">pom.xml</code>. Under <code class="language-plaintext highlighter-rouge">&lt;build&gt;</code> add the following lines:
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt">&lt;resources&gt;</span>
<span class="nt">&lt;resource&gt;</span>
<span class="nt">&lt;directory&gt;</span>${project.basedir}/src/main/resources<span class="nt">&lt;/directory&gt;</span>
<span class="nt">&lt;filtering&gt;</span>true<span class="nt">&lt;/filtering&gt;</span>
<span class="nt">&lt;/resource&gt;</span>
<span class="nt">&lt;/resources&gt;</span>
</code></pre></div> </div>
</li>
</ul>
<h3 id="12-test-the-rules">12. Test the rules</h3>
<ul>
<li>Testing rules is described in depth in <a href="pmd_userdocs_extending_testing.html">Testing your rules</a>.
<ul>
<li>Each rule has its own test class: Create a test class for your rule extending <code class="language-plaintext highlighter-rouge">PmdRuleTst</code>
<em>(see
<a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/test/java/net/sourceforge/pmd/lang/swift/rule/bestpractices/UnavailableFunctionTest.java"><code class="language-plaintext highlighter-rouge">UnavailableFunctionTest</code></a>
for example)</em></li>
<li>Create a category rule set for your language <em>(see
<a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/main/resources/category/swift/bestpractices.xml"><code class="language-plaintext highlighter-rouge">pmd-swift/src/main/resources/bestpractices.xml</code></a>
for example)</em></li>
<li>Place the test XML file with the test cases in the correct location</li>
<li>When executing the test class
<ul>
<li>this triggers the unit test to read the corresponding XML file with the rule test data
<em>(see
<a href="https://github.com/pmd/pmd/blob/main/pmd-swift/src/test/resources/net/sourceforge/pmd/lang/swift/rule/bestpractices/xml/UnavailableFunction.xml"><code class="language-plaintext highlighter-rouge">UnavailableFunction.xml</code></a>
for example)</em></li>
<li>This test XML file contains sample pieces of code which should trigger a specified number of
violations of this rule. The unit test will execute the rule on this piece of code, and verify
that the number of violations matches.</li>
</ul>
</li>
</ul>
</li>
<li>
<p>To verify the validity of all the created rulesets, create a subclass of <code class="language-plaintext highlighter-rouge">AbstractRuleSetFactoryTest</code>
(<em>see <code class="language-plaintext highlighter-rouge">RuleSetFactoryTest</code> in pmd-swift for example)</em>.
This will load all rulesets and verify, that all required attributes are provided.</p>
<p><em>Note:</em> You’ll need to add your ruleset to <code class="language-plaintext highlighter-rouge">categories.properties</code>, so that it can be found.</p>
</li>
</ul>
<h3 id="13-create-documentation-page">13. Create documentation page</h3>
<p>Finishing up your new language module by adding a page in the documentation. Create a new markdown file
<code class="language-plaintext highlighter-rouge">&lt;langId&gt;.md</code> in <code class="language-plaintext highlighter-rouge">docs/pages/pmd/languages/</code>. This file should have the following frontmatter:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
title: &lt;Language Name&gt;
permalink: pmd_languages_&lt;langId&gt;.html
last_updated: &lt;Month&gt; &lt;Year&gt; (&lt;PMD Version&gt;)
tags: [languages, PmdCapableLanguage, CpdCapableLanguage]
---
</code></pre></div></div>
<p>On this page, language specifics can be documented, e.g. when the language was first supported by PMD.
There is also the following Jekyll Include, that creates summary box for the language:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
{% include language_info.html name='&lt;Language Name&gt;' id='&lt;langId&gt;' implementation='&lt;langId&gt;::lang.&lt;langId&gt;.&lt;langId&gt;LanguageModule' supports_cpd=true supports_pmd=true %}
</code></pre></div></div>
<h2 id="optional-features">Optional features</h2>
<p>See <a href="pmd_devdocs_major_adding_new_language_javacc.html#optional-features">Optional features in JavaCC based languages</a>.</p>
<p>In order to implement these, most likely an AST needs to be developed first. The parse tree (CST, concrete
syntax tree) is not suitable to add methods such as <code class="language-plaintext highlighter-rouge">getSymbol()</code> to the node classes.</p>
<div class="tags">
<b>Tags: </b>
<a href="tag_devdocs.html" class="btn btn-outline-secondary navbar-btn cursorNorm" role="button">devdocs</a>
<a href="tag_extending.html" class="btn btn-outline-secondary navbar-btn cursorNorm" role="button">extending</a>
</div>
</div>
<footer>
<hr />
<div>
This documentation is written in markdown. <br />
If there is something missing or can be improved, edit this page on
github and create a PR:
<a
target="_blank"
href="https://github.com/pmd/pmd/blob/main/docs/pages/pmd/devdocs/major_contributions/adding_a_new_antlr_based_language.md"
role="button"
><i class="fab fa-github fa-lg"></i> Edit on GitHub</a
>
</div>
<hr />
<div class="row">
<div class="col-lg-12 footer">
&copy;2024 PMD Open Source Project. All rights
reserved. <br />
<span>Page last updated:</span>
December 2023 (7.0.0)<br /> Site last generated: Nov 21, 2024 <br />
<p>
<img src="images/logo/pmd-logo-70px.png" alt="PMD
logo"/>
</p>
</div>
</div>
</footer>
</div>
<!-- /.row -->
</div>
<!-- /.container -->
</div>
<!-- Sticky TOC column -->
<div class="toc-col">
<div id="toc"></div>
</div>
<!-- /.toc-container-wrapper -->
</div>
</div>
<script type="application/javascript" src="assets/jquery-3.5.1/jquery-3.5.1.min.js"></script>
<script type="application/javascript" src="assets/anchorjs-4.2.2/anchor.min.js"></script>
<script type="application/javascript" src="assets/navgoco-0.2.1/src/jquery.navgoco.min.js"></script>
<script type="application/javascript" src="assets/bootstrap-4.5.2-dist/js/bootstrap.bundle.min.js"></script>
<script type="application/javascript" src="assets/Simple-Jekyll-Search-1.0.8/dest/jekyll-search.js"></script>
<script type="application/javascript" src="assets/jekyll-table-of-contents/toc.js"></script>
<script type="application/javascript" src="js/tabstate.js"></script>
<script type="application/javascript" src="js/customscripts.js"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink