Ensure CPD uses tab width of 1 for tabs consistently

The columns that are reported by CPD were inconsistent across languages
before. A language like Java (using a JavaCC-based tokenizer) would use
a width of 8 for tabs, whereas a language like C# (using an Antlr-based
tokenizer) would use 1 instead.

This includes unit tests for most languages to ensure a tab character is
counted as 1. The configuration for JavaCC has been adjusted to respect
this as well.
This commit is contained in:
Maikel Steneker
2020-07-20 10:42:21 +02:00
parent 25405eb870
commit 6fb5ac59b9
45 changed files with 724 additions and 62 deletions

View File

@ -41,4 +41,9 @@ public class ObjectiveCTokenizerTest extends CpdTextComparisonTest {
public void testUnicodeCharInIdent() {
doTest("unicodeCharInIdent");
}
@Test
public void testTabWidth() {
doTest("tabWidth");
}
}

View File

@ -0,0 +1,9 @@
#import <Foundation/Foundation.h>
int main (int argc, const char * argv[])
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSLog (@"Hello, World!");
[pool drain];
return 0;
}

View File

@ -0,0 +1,49 @@
[Image] or [Truncated image[ Bcol Ecol
L3
[int] 1 3
[main] 5 8
[(] 10 10
[int] 11 13
[argc] 15 18
[,] 19 19
[const] 21 25
[char] 27 30
[*] 32 32
[argv] 34 37
[\[] 38 38
[\]] 39 39
[)] 40 40
L4
[{] 1 1
L5
[NSAutoreleasePool] 2 18
[*] 20 20
[pool] 21 24
[=] 26 26
[\[] 28 28
[\[] 29 29
[NSAutoreleasePool] 30 46
[alloc] 48 52
[\]] 53 53
[init] 55 58
[\]] 59 59
[;] 60 60
L6
[NSLog] 2 6
[(] 8 8
[@"Hello, World!"] 9 24
[)] 25 25
[;] 26 26
L7
[\[] 2 2
[pool] 3 6
[drain] 8 12
[\]] 13 13
[;] 14 14
L8
[return] 2 7
[0] 9 9
[;] 10 10
L9
[}] 1 1
EOF