phoedos/pmd
1
1
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
pmd/pmd_devdocs_major_adding_new_cpd_language.html

2162 lines
60 KiB
HTML
Raw Permalink Blame History

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="How to add a new language module with CPD support.">
<meta name="keywords" content="devdocsextending, ">
<title>How to add a new CPD language | PMD Source Code Analyzer</title>
<link rel="stylesheet" type="text/css" href="assets/fontawesome-free-5.15.4-web/css/all.min.css">
<link rel="stylesheet" type="text/css" href="assets/bootstrap-4.5.2-dist/css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="css/syntax.css">
<link rel="stylesheet" type="text/css" href="css/modern-business.css">
<link rel="stylesheet" type="text/css" href="css/customstyles.css">
<link rel="stylesheet" type="text/css" href="css/theme-green.css">
<link rel="stylesheet" type="text/css" href="css/pmd-customstyles.css">
<link rel="shortcut icon" href="images/logo/favicon.ico" type="image/x-icon">
<link rel="icon" href="images/logo/favicon.ico" type="image/x-icon">
<link rel="alternate" type="application/rss+xml" title="" href="feed.xml">
</head>
<body>
<!-- Content is offset by the height of the topnav bar. -->
<!-- There's already a padding-top rule in modern-business.css, but it apparently doesn't work on Firefox 60 and Chrome 67 -->
<div id="topbar-content-offset">
<!-- Navigation -->
<nav class="navbar navbar-expand-lg fixed-top navbar-dark">
<div class="container topnavlinks">
<a class="navbar-brand fas fa-home fa-lg" href="index.html">&nbsp;<span class="projectTitle"> PMD Source Code Analyzer Project</span></a>
<button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarSupportedContent">
<ul class="navbar-nav mr-auto mt-2 mt-lg-0"></ul>
<ul class="navbar-nav">
<!-- toggle sidebar button -->
<li class="nav-item"><a id="tg-sb-link" class="nav-link" href="#"><i id="tg-sb-icon" class="fas fa-toggle-on"></i> Nav</a></li>
<!-- entries without drop-downs appear here -->
<li class="nav-item"><a class="nav-link" href="https://github.com/pmd/pmd/releases/latest" target="_blank">Download</a></li>
<li class="nav-item"><a class="nav-link" href="https://github.com/pmd/pmd" target="_blank">Fork us on github</a></li>
<!-- entries with drop-downs appear here -->
<!-- conditional logic to control which topnav appears for the audience defined in the configuration file.-->
</ul>
<form class="form-inline my-2 my-lg-0">
<input class="form-control mr-sm-2" type="search" placeholder="search..." id="search-input">
<ul id="results-container"></ul>
</form>
</div>
</div>
</nav>
<!-- Page Content -->
<div class="container-toc-wrapper">
<div class="container">
<div class="col-lg-12">&nbsp;</div>
<!-- Content Row -->
<div class="row">
<!-- Sidebar Column -->
<div class="col-md-3" id="tg-sb-sidebar">
<ul id="mysidebar" class="nav">
<li class="sidebarTitle">PMD 7.8.0-SNAPSHOT</li>
<div class="sidebarTitleDate">Release date: 29-November-2024</div>
<li>
<a href="#">About</a>
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="pmd_release_notes.html">Release notes</a></li>
<li><a href="pmd_release_notes_pmd7.html">Release notes (PMD 7)</a></li>
<li><a href="pmd_about_help.html">Getting help</a></li>
<li><a href="pmd_about_release_policies.html">Release policies</a></li>
<li><a href="pmd_about_support_lifecycle.html">Support lifecycle</a></li>
</ul>
</li>
<li>
<a href="#">User Documentation</a>
<ul>
<li><a href="pmd_userdocs_migrating_to_pmd7.html">Migration Guide for PMD 7</a></li>
<li><a href="pmd_userdocs_installation.html">Installation and basic CLI usage</a></li>
<li><a href="pmd_userdocs_making_rulesets.html">Making rulesets</a></li>
<li><a href="pmd_userdocs_configuring_rules.html">Configuring rules</a></li>
<li><a href="pmd_userdocs_best_practices.html">Best practices</a></li>
<li><a href="pmd_userdocs_suppressing_warnings.html">Suppressing warnings</a></li>
<li><a href="pmd_userdocs_incremental_analysis.html">Incremental analysis</a></li>
<li><a href="pmd_userdocs_cli_reference.html">PMD CLI reference</a></li>
<li><a href="pmd_userdocs_report_formats.html">PMD Report formats</a></li>
<li><a href="pmd_userdocs_3rdpartyrulesets.html">3rd party rulesets</a></li>
<li class="subfolders">
<a href="#">CPD reference</a>
<ul>
<li><a href="pmd_userdocs_cpd.html">Copy-paste detection</a></li>
<li><a href="pmd_userdocs_cpd_report_formats.html">CPD Report formats</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Extending PMD</a>
<ul>
<li><a href="pmd_userdocs_extending_writing_rules_intro.html">Introduction to writing rules</a></li>
<li><a href="pmd_userdocs_extending_your_first_rule.html">Your first rule</a></li>
<li><a href="pmd_userdocs_extending_writing_xpath_rules.html">XPath rules</a></li>
<li><a href="pmd_userdocs_extending_writing_java_rules.html">Java rules</a></li>
<li><a href="pmd_userdocs_extending_designer_reference.html">Rule designer reference</a></li>
<li><a href="pmd_userdocs_extending_defining_properties.html">Defining rule properties</a></li>
<li><a href="pmd_userdocs_extending_rule_guidelines.html">Rule guidelines</a></li>
<li><a href="pmd_userdocs_extending_testing.html">Testing your rules</a></li>
<li><a href="pmd_userdocs_extending_ast_dump.html">Creating (XML) dump of the AST</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Tools / Integrations</a>
<ul>
<li><a href="pmd_userdocs_tools_maven.html">Maven PMD Plugin</a></li>
<li><a href="pmd_userdocs_tools_gradle.html">Gradle</a></li>
<li><a href="pmd_userdocs_tools_ant.html">Ant</a></li>
<li><a href="pmd_userdocs_tools_java_api.html">PMD Java API</a></li>
<li><a href="pmd_userdocs_tools_bld.html">bld PMD Extension</a></li>
<li><a href="pmd_userdocs_tools_ci.html">CI integrations</a></li>
<li><a href="pmd_userdocs_tools.html">Other Tools / Integrations</a></li>
</ul>
</li>
</ul>
</li>
<li>
<a href="#">Rule Reference</a>
<ul>
<li class="subfolders">
<a href="#">Apex Rules</a>
<ul>
<li><a href="pmd_rules_apex.html">Index</a></li>
<li><a href="pmd_rules_apex_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_apex_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_apex_design.html">Design</a></li>
<li><a href="pmd_rules_apex_documentation.html">Documentation</a></li>
<li><a href="pmd_rules_apex_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_apex_performance.html">Performance</a></li>
<li><a href="pmd_rules_apex_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">HTML Rules</a>
<ul>
<li><a href="pmd_rules_html.html">Index</a></li>
<li><a href="pmd_rules_html_bestpractices.html">Best Practices</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Java Rules</a>
<ul>
<li><a href="pmd_rules_java.html">Index</a></li>
<li><a href="pmd_rules_java_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_java_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_java_design.html">Design</a></li>
<li><a href="pmd_rules_java_documentation.html">Documentation</a></li>
<li><a href="pmd_rules_java_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_java_multithreading.html">Multithreading</a></li>
<li><a href="pmd_rules_java_performance.html">Performance</a></li>
<li><a href="pmd_rules_java_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Java Server Pages Rules</a>
<ul>
<li><a href="pmd_rules_jsp.html">Index</a></li>
<li><a href="pmd_rules_jsp_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_jsp_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_jsp_design.html">Design</a></li>
<li><a href="pmd_rules_jsp_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_jsp_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">JavaScript Rules</a>
<ul>
<li><a href="pmd_rules_ecmascript.html">Index</a></li>
<li><a href="pmd_rules_ecmascript_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_ecmascript_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_ecmascript_errorprone.html">Error Prone</a></li>
<li><a href="pmd_rules_ecmascript_performance.html">Performance</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Kotlin Rules</a>
<ul>
<li><a href="pmd_rules_kotlin.html">Index</a></li>
<li><a href="pmd_rules_kotlin_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_kotlin_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Maven POM Rules</a>
<ul>
<li><a href="pmd_rules_pom.html">Index</a></li>
<li><a href="pmd_rules_pom_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Modelica Rules</a>
<ul>
<li><a href="pmd_rules_modelica.html">Index</a></li>
<li><a href="pmd_rules_modelica_bestpractices.html">Best Practices</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">PLSQL Rules</a>
<ul>
<li><a href="pmd_rules_plsql.html">Index</a></li>
<li><a href="pmd_rules_plsql_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_plsql_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_plsql_design.html">Design</a></li>
<li><a href="pmd_rules_plsql_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Salesforce Visualforce Rules</a>
<ul>
<li><a href="pmd_rules_visualforce.html">Index</a></li>
<li><a href="pmd_rules_visualforce_security.html">Security</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Scala Rules</a>
<ul>
<li><a href="pmd_rules_scala.html">Index</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Swift Rules</a>
<ul>
<li><a href="pmd_rules_swift.html">Index</a></li>
<li><a href="pmd_rules_swift_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_swift_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Velocity Template Language (VTL) Rules</a>
<ul>
<li><a href="pmd_rules_velocity.html">Index</a></li>
<li><a href="pmd_rules_velocity_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_velocity_design.html">Design</a></li>
<li><a href="pmd_rules_velocity_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">WSDL Rules</a>
<ul>
<li><a href="pmd_rules_wsdl.html">Index</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">XML Rules</a>
<ul>
<li><a href="pmd_rules_xml.html">Index</a></li>
<li><a href="pmd_rules_xml_bestpractices.html">Best Practices</a></li>
<li><a href="pmd_rules_xml_errorprone.html">Error Prone</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">XSL Rules</a>
<ul>
<li><a href="pmd_rules_xsl.html">Index</a></li>
<li><a href="pmd_rules_xsl_codestyle.html">Code Style</a></li>
<li><a href="pmd_rules_xsl_performance.html">Performance</a></li>
</ul>
</li>
</ul>
</li>
<li>
<a href="#">Language-Specific Documentation</a>
<ul>
<li><a href="pmd_languages_index.html">Overview</a></li>
<li><a href="pmd_languages_configuration.html">Language configuration</a></li>
<li><a href="pmd_languages_apex.html">Apex</a></li>
<li><a href="pmd_languages_cpp.html">C/C++</a></li>
<li><a href="pmd_languages_cs.html">C#</a></li>
<li><a href="pmd_languages_coco.html">Coco</a></li>
<li><a href="pmd_languages_dart.html">Dart</a></li>
<li><a href="pmd_languages_fortran.html">Fortran</a></li>
<li><a href="pmd_languages_gherkin.html">Gherkin</a></li>
<li><a href="pmd_languages_go.html">Go</a></li>
<li><a href="pmd_languages_html.html">HTML</a></li>
<li><a href="pmd_languages_java.html">Java</a></li>
<li><a href="pmd_languages_js_ts.html">JavaScript / TypeScript</a></li>
<li><a href="pmd_languages_jsp.html">JSP</a></li>
<li><a href="pmd_languages_julia.html">Julia</a></li>
<li><a href="pmd_languages_kotlin.html">Kotlin</a></li>
<li><a href="pmd_languages_lua.html">Lua</a></li>
<li><a href="pmd_languages_matlab.html">Matlab</a></li>
<li><a href="pmd_languages_modelica.html">Modelica</a></li>
<li><a href="pmd_languages_objectivec.html">Objective-C</a></li>
<li><a href="pmd_languages_perl.html">Perl</a></li>
<li><a href="pmd_languages_php.html">PHP</a></li>
<li><a href="pmd_languages_plsql.html">PLSQL</a></li>
<li><a href="pmd_languages_python.html">Python</a></li>
<li><a href="pmd_languages_ruby.html">Ruby</a></li>
<li><a href="pmd_languages_scala.html">Scala</a></li>
<li><a href="pmd_languages_swift.html">Swift</a></li>
<li><a href="pmd_languages_tsql.html">T-SQL</a></li>
<li><a href="pmd_languages_visualforce.html">Visualforce</a></li>
<li><a href="pmd_languages_velocity.html">Velocity Template Language (VTL)</a></li>
<li><a href="pmd_languages_xml.html">XML and XML dialects</a></li>
</ul>
</li>
<li>
<a href="#">Developer Documentation</a>
<ul>
<li><a href="pmd_devdocs_development.html">Developer resources</a></li>
<li><a href="pmd_devdocs_building.html">Building PMD from source</a></li>
<li><a href="https://github.com/pmd/pmd/blob/main/CONTRIBUTING.md" target="_blank">Contributing</a></li>
<li><a href="pmd_devdocs_writing_documentation.html">Writing documentation</a></li>
<li><a href="pmd_devdocs_roadmap.html">Roadmap</a></li>
<li><a href="pmd_devdocs_how_pmd_works.html">How PMD works</a></li>
<li><a href="pmd_devdocs_pmdtester.html">Pmdtester</a></li>
<li><a href="pmd_devdocs_rule_deprecation_policy.html">Rule Deprecation Policy</a></li>
<li class="subfolders">
<a href="#">Major contributions</a>
<ul>
<li><a href="pmd_devdocs_major_rule_guidelines.html">Rule Guidelines</a></li>
<li><a href="pmd_devdocs_major_adding_new_language_javacc.html">Adding a new language (JavaCC)</a></li>
<li><a href="pmd_devdocs_major_adding_new_language_antlr.html">Adding a new language (ANTLR)</a></li>
<li class="active"><a href="pmd_devdocs_major_adding_new_cpd_language.html">Adding a new CPD language</a></li>
</ul>
</li>
<li class="subfolders">
<a href="#">Experimental features</a>
<ul>
<li><a href="tag_experimental.html">List of experimental Features</a></li>
</ul>
</li>
</ul>
</li>
<li>
<a href="#">Project documentation</a>
<ul>
<li class="subfolders">
<a href="#">Trivia about PMD</a>
<ul>
<li><a href="pmd_projectdocs_trivia_news.html">PMD in the press</a></li>
<li><a href="pmd_projectdocs_trivia_products.html">Products & books related to PMD</a></li>
<li><a href="pmd_projectdocs_trivia_similarprojects.html">Similar projects</a></li>
<li><a href="pmd_projectdocs_trivia_meaning.html">What does 'PMD' mean?</a></li>
</ul>
</li>
<li><a href="pmd_projectdocs_logo.html">Logo</a></li>
<li><a href="pmd_projectdocs_faq.html">FAQ</a></li>
<li><a href="license.html">License</a></li>
<li><a href="pmd_projectdocs_credits.html">Credits</a></li>
<li><a href="pmd_release_notes_old.html">Old release notes</a></li>
<li><a href="pmd_projectdocs_decisions.html">Decisions</a></li>
<li class="subfolders">
<a href="#">Project management</a>
<ul>
<li><a href="pmd_projectdocs_committers_infrastructure.html">Infrastructure</a></li>
<li><a href="pmd_projectdocs_committers_releasing.html">Release process</a></li>
<li><a href="pmd_projectdocs_committers_merging_pull_requests.html">Merging pull requests</a></li>
<li><a href="pmd_projectdocs_committers_main_landing_page.html">Main Landing page</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<!-- Content Column -->
<div class="col-md-9" id="tg-sb-content">
<header>
<div class="row">
<div class="col-lg-12">
<a href="./" role="button"
><i class="fa fa-home fa-lg"></i
></a>
» How to add a new CPD language
<a
target="_blank"
href="https://github.com/pmd/pmd/blob/main/docs/pages/pmd/devdocs/major_contributions/adding_new_cpd_language.md"
class="float-right"
role="button"
><i class="fab fa-github fa-lg"></i> Edit on GitHub</a
>
</div>
</div>
<hr />
</header>
<div class="post-header">
<h1 class="post-title-main">How to add a new CPD language</h1>
</div>
<div class="post-content" data-github-edit-url="https://github.com/pmd/pmd/blob/main/docs/pages/pmd/devdocs/major_contributions/adding_new_cpd_language.md">
<div class="summary">How to add a new language module with CPD support.</div>
<details id="inline-toc-details">
<summary>Table of Contents</summary>
<div id="inline-toc"><!-- empty, move TOC here when screen size too small --></div>
</details>
<h2 id="adding-support-for-a-cpd-language">Adding support for a CPD language</h2>
<p>CPD works generically on the tokens produced by a <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/CpdLexer.html#"><code>CpdLexer</code></a>.
To add support for a new language, the crucial piece is writing a CpdLexer that
splits the source file into the tokens specific to your language. Thankfully you
can use a stock <a href="https://github.com/antlr/grammars-v4">Antlr grammar</a> or JavaCC
grammar to generate a lexer for you. If you cannot use a lexer generator, for
instance because you are wrapping a lexer from another library, it is still relatively
easy to implement the Tokenizer interface.</p>
<p>Use the following guide to set up a new language module that supports CPD.</p>
<ol>
<li>Create a new Maven module for your language. You can take <a href="https://github.com/pmd/pmd/tree/main/pmd-go/pom.xml">the Golang module</a> as an example.
<ul>
<li>Make sure to add your new module to the parent pom as <code class="language-plaintext highlighter-rouge">&lt;module&gt;</code> entry, so that it is built alongside the
other languages.</li>
<li>Also add your new module to the dependencies list in “pmd-languages-deps/pom.xml”, so that the new language
is automatically available in the binary distribution (pmd-dist).</li>
</ul>
</li>
<li>Implement a <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/CpdLexer.html#"><code>CpdLexer</code></a>.
<ul>
<li>
<p>For Antlr grammars you can take the grammar from <a href="https://github.com/antlr/grammars-v4">antlr/grammars-v4</a> and place it in <code class="language-plaintext highlighter-rouge">src/main/antlr4</code> followed by the package name of the language. You then need to call the appropriate ant wrapper to generate
the lexer from the grammar. To do so, edit <code class="language-plaintext highlighter-rouge">pom.xml</code> (eg like <a href="https://github.com/pmd/pmd/tree/main/pmd-go/pom.xml">the Golang module</a>).
Once that is done, <code class="language-plaintext highlighter-rouge">mvn generate-sources</code> should generate the lexer sources for you.</p>
<p>You can now implement a CpdLexer, for instance by extending <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/impl/AntlrCpdLexer.html#"><code>AntlrCpdLexer</code></a>. The following reproduces the Go implementation:
```java
// mind the package convention if you are going to make a PR
package net.sourceforge.pmd.lang.go.cpd;</p>
</li>
</ul>
<p>public class GoCpdLexer extends AntlrCpdLexer {</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> @Override
protected Lexer getLexerForSource(CharStream charStream) {
return new GolangLexer(charStream);
} } ```
</code></pre></div> </div>
<ul>
<li>If your language is case-insensitive, then you might want to overwrite <code class="language-plaintext highlighter-rouge">getImage(AntlrToken)</code>. There you can
change each token e.g. into uppercase, so that CPD sees the same strings and can find duplicates even when
the casing differs. See <a href="https://docs.pmd-code.org/apidocs/pmd-tsql/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/tsql/cpd/TSqlCpdLexer.html#"><code>TSqlCpdLexer</code></a> for an example. You will also need a
“CaseChangingCharStream”, so that antlr itself is case-insensitive.</li>
<li>For JavaCC grammars, place your grammar in <code class="language-plaintext highlighter-rouge">etc/grammar</code> and edit the <code class="language-plaintext highlighter-rouge">pom.xml</code> like the <a href="https://github.com/pmd/pmd/blob/main/pmd-python/pom.xml">Python implementation</a> does.
You can then subclass <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/impl/JavaccCpdLexer.html#"><code>JavaccCpdLexer</code></a> instead of AntlrCpdLexer.</li>
<li>If your JavaCC based language is case-insensitive (option <code class="language-plaintext highlighter-rouge">IGNORE_CASE=true</code>), then you need to implement
<a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/ast/impl/javacc/JavaccTokenDocument.TokenDocumentBehavior.html#"><code>JavaccTokenDocument.TokenDocumentBehavior</code></a>, which can change each token
e.g. into uppercase. See <a href="https://docs.pmd-code.org/apidocs/pmd-plsql/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/plsql/ast/PLSQLParser.html#"><code>PLSQLParser</code></a> for an example.</li>
<li>For any other scenario just implement the interface however you can. Look at the Scala or Apex module for existing implementations.</li>
</ul>
</li>
<li>
<p>Create a <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/Language.html#"><code>Language</code></a> implementation, and make it implement <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/CpdCapableLanguage.html#"><code>CpdCapableLanguage</code></a>.
If your language only supports CPD, then you can subclass <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/impl/CpdOnlyLanguageModuleBase.html#"><code>CpdOnlyLanguageModuleBase</code></a> to get going:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1">// mind the package convention if you are going to make a PR</span>
<span class="kn">package</span> <span class="nn">net.sourceforge.pmd.lang.go</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">GoLanguageModule</span> <span class="kd">extends</span> <span class="nc">CpdOnlyLanguageModuleBase</span> <span class="o">{</span>
<span class="c1">// A public noarg constructor is required.</span>
<span class="kd">public</span> <span class="nf">GoLanguageModule</span><span class="o">()</span> <span class="o">{</span>
<span class="kd">super</span><span class="o">(</span><span class="nc">LanguageMetadata</span><span class="o">.</span><span class="na">withId</span><span class="o">(</span><span class="s">"go"</span><span class="o">).</span><span class="na">name</span><span class="o">(</span><span class="s">"Go"</span><span class="o">).</span><span class="na">extensions</span><span class="o">(</span><span class="s">"go"</span><span class="o">));</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="nc">Tokenizer</span> <span class="nf">createCpdLexer</span><span class="o">(</span><span class="nc">LanguagePropertyBundle</span> <span class="n">bundle</span><span class="o">)</span> <span class="o">{</span>
<span class="c1">// This method should return an instance of the CpdLexer you created.</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">GoCpdLexer</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div> </div>
<p>To make PMD find the language module at runtime, write the fully-qualified name of your language class into the file <code class="language-plaintext highlighter-rouge">src/main/resources/META-INF/services/net.sourceforge.pmd.lang.Language</code>.</p>
<p>At this point the new language module should be available in <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/LanguageRegistry.html#CPD"><code>CPD</code></a> and usable by CPD like any other language.</p>
</li>
<li>
<p>Update the test that asserts the list of supported languages by updating the <code class="language-plaintext highlighter-rouge">SUPPORTED_LANGUAGES</code> constant in <a href="https://github.com/pmd/pmd/blob/main/pmd-dist/src/test/java/net/sourceforge/pmd/dist/BinaryDistributionIT.java">BinaryDistributionIT</a>.</p>
</li>
<li>
<p>Add some tests for your CpdLexer by following the <a href="#testing-your-implementation">section below</a>.</p>
</li>
<li>
<p>Finishing up your new language module by adding a page in the documentation. Create a new markdown file
<code class="language-plaintext highlighter-rouge">&lt;langId&gt;.md</code> in <code class="language-plaintext highlighter-rouge">docs/pages/pmd/languages/</code>. This file should have the following frontmatter:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
title: &lt;Language Name&gt;
permalink: pmd_languages_&lt;langId&gt;.html
last_updated: &lt;Month&gt; &lt;Year&gt; (&lt;PMD Version&gt;)
tags: [languages, CpdCapableLanguage]
---
</code></pre></div> </div>
<p>On this page, language specifics can be documented, e.g. when the language was first supported by PMD.
There is also the following Jekyll Include, that creates summary box for the language:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
{% include language_info.html name='&lt;Language Name&gt;' id='&lt;langId&gt;' implementation='&lt;langId&gt;::lang.&lt;langId&gt;.&lt;langId&gt;LanguageModule' supports_cpd=true %}
</code></pre></div> </div>
</li>
</ol>
<h3 id="declaring-cpdlexer-options">Declaring CpdLexer options</h3>
<p>To make the CpdLexer configurable, first define some property descriptors using
<a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/properties/PropertyFactory.html#"><code>PropertyFactory</code></a>. Look at <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/CpdLanguageProperties.html#"><code>CpdLanguageProperties</code></a>
for some predefined ones which you can reuse (prefer reusing property descriptors if you can).
You need to override <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/Language.html#newPropertyBundle()"><code>newPropertyBundle</code></a>
and call <code class="language-plaintext highlighter-rouge">definePropertyDescriptor</code> to register the descriptors.
After that you can access the values of the properties from the parameter
of <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/CpdCapableLanguage.html#createCpdTokenizer(net.sourceforge.pmd.lang.LanguagePropertyBundle)"><code>createCpdTokenizer</code></a>.</p>
<p>To implement simple token filtering, you can use <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/impl/BaseTokenFilter.html#"><code>BaseTokenFilter</code></a>
as a base class, or another base class in <a href="https://docs.pmd-code.org/apidocs/pmd-core/7.8.0-SNAPSHOT/net/sourceforge/pmd/cpd/impl/package-summary.html#"><code>net.sourceforge.pmd.cpd.impl</code></a>.
Take a look at the <a href="https://github.com/pmd/pmd/blob/main/pmd-kotlin/src/main/java/net/sourceforge/pmd/lang/kotlin/cpd/KotlinCpdLexer.java">Kotlin token filter implementation</a>, or the <a href="https://github.com/pmd/pmd/blob/master/pmd-java/src/main/java/net/sourceforge/pmd/lang/java/cpd/JavaCpdLexer.java">Java one</a>.</p>
<h3 id="testing-your-implementation">Testing your implementation</h3>
<p>Add a Maven dependency on <code class="language-plaintext highlighter-rouge">pmd-lang-test</code> (scope <code class="language-plaintext highlighter-rouge">test</code>) in your <code class="language-plaintext highlighter-rouge">pom.xml</code>.
This contains utilities to test your CpdLexer.</p>
<p>Create a test class extending from <a href="https://docs.pmd-code.org/apidocs/pmd-lang-test/7.8.0-SNAPSHOT/net/sourceforge/pmd/lang/test/cpd/CpdTextComparisonTest.html#"><code>CpdTextComparisonTest</code></a>.
To add tests, you need to write regular JUnit <code class="language-plaintext highlighter-rouge">@Test</code>-annotated methods, and
call the method <code class="language-plaintext highlighter-rouge">doTest</code> with the name of the test file.</p>
<p>For example, for the Dart language:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">package</span> <span class="nn">net.sourceforge.pmd.lang.dart.cpd</span><span class="o">;</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">DartTokenizerTest</span> <span class="kd">extends</span> <span class="nc">CpdTextComparisonTest</span> <span class="o">{</span>
<span class="cm">/**********************************
Implementation of the superclass
***********************************/</span>
<span class="kd">public</span> <span class="nf">DartTokenizerTest</span><span class="o">()</span> <span class="o">{</span>
<span class="kd">super</span><span class="o">(</span><span class="s">"dart"</span><span class="o">,</span> <span class="s">".dart"</span><span class="o">);</span> <span class="c1">// the ID of the language, then the file extension used by test files</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">protected</span> <span class="nc">String</span> <span class="nf">getResourcePrefix</span><span class="o">()</span> <span class="o">{</span>
<span class="c1">// "testdata" is the default value, you don't need to override.</span>
<span class="c1">// This specifies that you should place the test files in</span>
<span class="c1">// src/test/resources/net/sourceforge/pmd/lang/dart/cpd/testdata</span>
<span class="k">return</span> <span class="s">"testdata"</span><span class="o">;</span>
<span class="o">}</span>
<span class="cm">/**************
Test methods
***************/</span>
<span class="nd">@Test</span> <span class="c1">// don't forget the JUnit annotation</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">testLiterals</span><span class="o">()</span> <span class="o">{</span>
<span class="c1">// This will look for a file named literals.dart</span>
<span class="c1">// in the directory identified by getResourcePrefix,</span>
<span class="c1">// tokenize it, then compare the result against a baseline</span>
<span class="c1">// literals.txt file in the same directory</span>
<span class="c1">// If the baseline file does not exist, it is created automatically</span>
<span class="n">doTest</span><span class="o">(</span><span class="s">"literals"</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<div class="tags">
<b>Tags: </b>
<a href="tag_devdocs.html" class="btn btn-outline-secondary navbar-btn cursorNorm" role="button">devdocs</a>
<a href="tag_extending.html" class="btn btn-outline-secondary navbar-btn cursorNorm" role="button">extending</a>
</div>
</div>
<footer>
<hr />
<div>
This documentation is written in markdown. <br />
If there is something missing or can be improved, edit this page on
github and create a PR:
<a
target="_blank"
href="https://github.com/pmd/pmd/blob/main/docs/pages/pmd/devdocs/major_contributions/adding_new_cpd_language.md"
role="button"
><i class="fab fa-github fa-lg"></i> Edit on GitHub</a
>
</div>
<hr />
<div class="row">
<div class="col-lg-12 footer">
&copy;2024 PMD Open Source Project. All rights
reserved. <br />
<span>Page last updated:</span>
June 2024 (7.3.0)<br /> Site last generated: Nov 21, 2024 <br />
<p>
<img src="images/logo/pmd-logo-70px.png" alt="PMD
logo"/>
</p>
</div>
</div>
</footer>
</div>
<!-- /.row -->
</div>
<!-- /.container -->
</div>
<!-- Sticky TOC column -->
<div class="toc-col">
<div id="toc"></div>
</div>
<!-- /.toc-container-wrapper -->
</div>
</div>
<script type="application/javascript" src="assets/jquery-3.5.1/jquery-3.5.1.min.js"></script>
<script type="application/javascript" src="assets/anchorjs-4.2.2/anchor.min.js"></script>
<script type="application/javascript" src="assets/navgoco-0.2.1/src/jquery.navgoco.min.js"></script>
<script type="application/javascript" src="assets/bootstrap-4.5.2-dist/js/bootstrap.bundle.min.js"></script>
<script type="application/javascript" src="assets/Simple-Jekyll-Search-1.0.8/dest/jekyll-search.js"></script>
<script type="application/javascript" src="assets/jekyll-table-of-contents/toc.js"></script>
<script type="application/javascript" src="js/tabstate.js"></script>
<script type="application/javascript" src="js/customscripts.js"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink