vpp/doxygen/filter_c.py

88 lines
4.2 KiB
Python
Raw Normal View History

#!/usr/bin/env python3
# Copyright (c) 2016 Comcast Cable Communications Management, LLC.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at:
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Filter for .c files to make various preprocessor tricks Doxygenish
VPP-221 CLI auto-documentation infrastructure As a step before Doxygen, extract CLI-related struct initializers from the code and parse that into a summary of the CLI commands available with the provided help text, such as it is. At the moment this only renders this into an indexed Markdown file that Doxygen then picks up but later we can use this information to enrich the existing VLIB_CLI_COMMAND macro documentor as well as provide runtime documentation to VPP that is stored on disk outside the binary image. Additionally support a comment block immediately prior to VLIB_CLI_COMMAND CLI command definitions in the form /*? ... ?*/ that can be used to include long-form documentation without having it compiled into VPP. Examples of documenting CLI commands can be found in vlib/vlib/unix/cli.c which, whilst not perfect, should provide a starting point. Screen captures of sample output can be seen at https://chrisy.flirble.org/vpp/doxy-cli-example.png and https://chrisy.flirble.org/vpp/doxy-cli-index.png . Next, shift the Doxygen root makefile targets to their own Makefile. The primary reason for this is that the siphon targets do dependency tracking which means it needs to generate those dependencies whenever make is run; that is pointless if we're not going to generate any documentation. This includes the package dependencies since they since they sometimes unnecessarily interfere with the code build in some cases at the moment; later we will look to building a Python venv to host the Python modules we use. One final remark: In future we may consider deprecating .long_help in the VLIB_CLI_COMMAND structure entirely but add perhaps .usage_help. .short_help would be reserved for a summary of the command function and .usage_help provide the syntax of that command. These changes would provide great semantic value to the automaticly generated CLI documentation. I could also see having .long_help replaced by a mechanism that reads it from disk at runtime with a rudimentary Markdown/Doxygen filter so that we can use the same text that is used in the published documentation. Change-Id: I80d6fe349b47dce649fa77d21ffec0ddb45c7bbf Signed-off-by: Chris Luke <chrisy@flirble.org>
2016-07-25 16:38:11 -04:00
import os, sys, re
if len(sys.argv) < 2:
sys.stderr.write("Usage: %s <filename>\n" % (sys.argv[0]))
sys.exit(1)
VPP-221 CLI auto-documentation infrastructure As a step before Doxygen, extract CLI-related struct initializers from the code and parse that into a summary of the CLI commands available with the provided help text, such as it is. At the moment this only renders this into an indexed Markdown file that Doxygen then picks up but later we can use this information to enrich the existing VLIB_CLI_COMMAND macro documentor as well as provide runtime documentation to VPP that is stored on disk outside the binary image. Additionally support a comment block immediately prior to VLIB_CLI_COMMAND CLI command definitions in the form /*? ... ?*/ that can be used to include long-form documentation without having it compiled into VPP. Examples of documenting CLI commands can be found in vlib/vlib/unix/cli.c which, whilst not perfect, should provide a starting point. Screen captures of sample output can be seen at https://chrisy.flirble.org/vpp/doxy-cli-example.png and https://chrisy.flirble.org/vpp/doxy-cli-index.png . Next, shift the Doxygen root makefile targets to their own Makefile. The primary reason for this is that the siphon targets do dependency tracking which means it needs to generate those dependencies whenever make is run; that is pointless if we're not going to generate any documentation. This includes the package dependencies since they since they sometimes unnecessarily interfere with the code build in some cases at the moment; later we will look to building a Python venv to host the Python modules we use. One final remark: In future we may consider deprecating .long_help in the VLIB_CLI_COMMAND structure entirely but add perhaps .usage_help. .short_help would be reserved for a summary of the command function and .usage_help provide the syntax of that command. These changes would provide great semantic value to the automaticly generated CLI documentation. I could also see having .long_help replaced by a mechanism that reads it from disk at runtime with a rudimentary Markdown/Doxygen filter so that we can use the same text that is used in the published documentation. Change-Id: I80d6fe349b47dce649fa77d21ffec0ddb45c7bbf Signed-off-by: Chris Luke <chrisy@flirble.org>
2016-07-25 16:38:11 -04:00
replace_patterns = [
# Search for VLIB_CLI_COMMAND, extract its parameters and add a docblock for it
( re.compile("(?P<m>VLIB_CLI_COMMAND)\s*[(](?P<name>[a-zA-Z0-9_]+)[)]"),
r"/** @brief (@em constructor) \g<m> (\g<name>) */ vlib_cli_command_t \g<name>"),
( re.compile("(?P<m>VLIB_CLI_COMMAND)\s*[(](?P<name>[a-zA-Z0-9_]+),\s*(?P<qual>[^)]*)[)]"),
r"/** @brief (@em constructor) \g<m> (\g<name>) */ \g<qual> vlib_cli_command_t \g<name>"),
# Search for VLIB_REGISTER_NODE, extract its parameters and add a docblock for it
( re.compile("(?P<m>VLIB_REGISTER_NODE)\s*[(](?P<name>[a-zA-Z0-9_]+)[)]"),
r"/** @brief (@em constructor) \g<m> (\g<name>) */ vlib_node_registration_t \g<name>"),
( re.compile("(?P<m>VLIB_REGISTER_NODE)\s*[(](?P<name>[a-zA-Z0-9_]+),\s*(?P<qual>[^)]*)[)]"),
r"/** @brief (@em constructor) \g<m> (\g<name>) */ \g<qual> vlib_node_registration_t \g<name>"),
# Search for VLIB_INIT_FUNCTION, extract its parameter and add a docblock for it
( re.compile("(?P<m>VLIB_INIT_FUNCTION)\s*[(](?P<name>[a-zA-Z0-9_]+)[)]"),
r"/** @brief (@em constructor) \g<m> (@ref \g<name>) */ vlib_init_function_t * _vlib_init_function_\g<name>"),
( re.compile("(?P<m>VLIB_DECLARE_INIT_FUNCTION)\s*[(](?P<name>[a-zA-Z0-9_]+)[)]"),
r"/** @brief (@em constructor) \g<m> (@ref \g<name>) */ vlib_init_function_t * _vlib_init_function_\g<name>"),
# Search for VLIB_LOOP_ENTER_FUNCTION, extract the parameters and add a docblock for it
( re.compile("(?P<m>VLIB_MAIN_LOOP_ENTER_FUNCTION)\s*[(](?P<name>[a-zA-Z0-9_]+)(,[^)]*)?[)]"),
r"/** @brief (@em constructor) \g<m> (@ref \g<name>) */ _vlib_main_loop_enter_\g<name>"),
( re.compile("(?P<m>VLIB_MAIN_LOOP_EXIT_FUNCTION)\s*[(](?P<name>[a-zA-Z0-9_]+)(,[^)]*)?[)]"),
r"/** @brief (@em constructor) \g<m> (@ref \g<name>) */ _vlib_main_loop_exit_\g<name>"),
# Search for VLIB_CONFIG_FUNCTION, extract the parameters and add a docblock for it
( re.compile("(?P<m>VLIB_CONFIG_FUNCTION)\s*[(](?P<name>[a-zA-Z0-9_]+),\s*(?P<n>\"[^\"]+\")(,[^)]*)?[)]"),
r"/** @brief (@em constructor) \g<m> (\g<name>, \g<n>) */ vlib_config_function_runtime_t _vlib_config_function_\g<name>"),
( re.compile("(?P<m>VLIB_EARLY_CONFIG_FUNCTION)\s*[(](?P<name>[a-zA-Z0-9_]+),\s*(?P<n>\"[^\"]+\")(,[^)]*)?[)]"),
r"/** @brief (@em constructor) \g<m> (\g<name>, \g<n>) */ vlib_config_function_runtime_t _vlib_config_function_\g<name>"),
# Search for "format_thing" and "unformat_thing" when used as a function pointer and add parens
( re.compile("(?P<pre>(^|,)\s*)(?P<name>(un)?format_[a-zA-Z0-9_]+)(?P<post>\s*(,|$))"),
r"\g<pre>\g<name>()\g<post>" ),
# Search for CLIB_PAD_FROM_TO(...); and replace with padding
# #define CLIB_PAD_FROM_TO(from,to) u8 pad_##from[(to) - (from)]
( re.compile("(?P<m>CLIB_PAD_FROM_TO)\s*[(](?P<from>[^,]+),\s*(?P<to>[^)]+)[)]"),
r"/** Padding. */ u8 pad_\g<from>[(\g<to>) - (\g<from>)]" ),
]
VPP-221 CLI auto-documentation infrastructure As a step before Doxygen, extract CLI-related struct initializers from the code and parse that into a summary of the CLI commands available with the provided help text, such as it is. At the moment this only renders this into an indexed Markdown file that Doxygen then picks up but later we can use this information to enrich the existing VLIB_CLI_COMMAND macro documentor as well as provide runtime documentation to VPP that is stored on disk outside the binary image. Additionally support a comment block immediately prior to VLIB_CLI_COMMAND CLI command definitions in the form /*? ... ?*/ that can be used to include long-form documentation without having it compiled into VPP. Examples of documenting CLI commands can be found in vlib/vlib/unix/cli.c which, whilst not perfect, should provide a starting point. Screen captures of sample output can be seen at https://chrisy.flirble.org/vpp/doxy-cli-example.png and https://chrisy.flirble.org/vpp/doxy-cli-index.png . Next, shift the Doxygen root makefile targets to their own Makefile. The primary reason for this is that the siphon targets do dependency tracking which means it needs to generate those dependencies whenever make is run; that is pointless if we're not going to generate any documentation. This includes the package dependencies since they since they sometimes unnecessarily interfere with the code build in some cases at the moment; later we will look to building a Python venv to host the Python modules we use. One final remark: In future we may consider deprecating .long_help in the VLIB_CLI_COMMAND structure entirely but add perhaps .usage_help. .short_help would be reserved for a summary of the command function and .usage_help provide the syntax of that command. These changes would provide great semantic value to the automaticly generated CLI documentation. I could also see having .long_help replaced by a mechanism that reads it from disk at runtime with a rudimentary Markdown/Doxygen filter so that we can use the same text that is used in the published documentation. Change-Id: I80d6fe349b47dce649fa77d21ffec0ddb45c7bbf Signed-off-by: Chris Luke <chrisy@flirble.org>
2016-07-25 16:38:11 -04:00
filename = sys.argv[1]
cwd = os.getcwd()
if filename[0:len(cwd)] == cwd:
filename = filename[len(cwd):]
if filename[0] == "/":
filename = filename[1:]
with open(filename) as fd:
line_num = 0
for line in fd:
VPP-221 CLI auto-documentation infrastructure As a step before Doxygen, extract CLI-related struct initializers from the code and parse that into a summary of the CLI commands available with the provided help text, such as it is. At the moment this only renders this into an indexed Markdown file that Doxygen then picks up but later we can use this information to enrich the existing VLIB_CLI_COMMAND macro documentor as well as provide runtime documentation to VPP that is stored on disk outside the binary image. Additionally support a comment block immediately prior to VLIB_CLI_COMMAND CLI command definitions in the form /*? ... ?*/ that can be used to include long-form documentation without having it compiled into VPP. Examples of documenting CLI commands can be found in vlib/vlib/unix/cli.c which, whilst not perfect, should provide a starting point. Screen captures of sample output can be seen at https://chrisy.flirble.org/vpp/doxy-cli-example.png and https://chrisy.flirble.org/vpp/doxy-cli-index.png . Next, shift the Doxygen root makefile targets to their own Makefile. The primary reason for this is that the siphon targets do dependency tracking which means it needs to generate those dependencies whenever make is run; that is pointless if we're not going to generate any documentation. This includes the package dependencies since they since they sometimes unnecessarily interfere with the code build in some cases at the moment; later we will look to building a Python venv to host the Python modules we use. One final remark: In future we may consider deprecating .long_help in the VLIB_CLI_COMMAND structure entirely but add perhaps .usage_help. .short_help would be reserved for a summary of the command function and .usage_help provide the syntax of that command. These changes would provide great semantic value to the automaticly generated CLI documentation. I could also see having .long_help replaced by a mechanism that reads it from disk at runtime with a rudimentary Markdown/Doxygen filter so that we can use the same text that is used in the published documentation. Change-Id: I80d6fe349b47dce649fa77d21ffec0ddb45c7bbf Signed-off-by: Chris Luke <chrisy@flirble.org>
2016-07-25 16:38:11 -04:00
line_num += 1
str = line[:-1] # filter \n
VPP-221 CLI auto-documentation infrastructure As a step before Doxygen, extract CLI-related struct initializers from the code and parse that into a summary of the CLI commands available with the provided help text, such as it is. At the moment this only renders this into an indexed Markdown file that Doxygen then picks up but later we can use this information to enrich the existing VLIB_CLI_COMMAND macro documentor as well as provide runtime documentation to VPP that is stored on disk outside the binary image. Additionally support a comment block immediately prior to VLIB_CLI_COMMAND CLI command definitions in the form /*? ... ?*/ that can be used to include long-form documentation without having it compiled into VPP. Examples of documenting CLI commands can be found in vlib/vlib/unix/cli.c which, whilst not perfect, should provide a starting point. Screen captures of sample output can be seen at https://chrisy.flirble.org/vpp/doxy-cli-example.png and https://chrisy.flirble.org/vpp/doxy-cli-index.png . Next, shift the Doxygen root makefile targets to their own Makefile. The primary reason for this is that the siphon targets do dependency tracking which means it needs to generate those dependencies whenever make is run; that is pointless if we're not going to generate any documentation. This includes the package dependencies since they since they sometimes unnecessarily interfere with the code build in some cases at the moment; later we will look to building a Python venv to host the Python modules we use. One final remark: In future we may consider deprecating .long_help in the VLIB_CLI_COMMAND structure entirely but add perhaps .usage_help. .short_help would be reserved for a summary of the command function and .usage_help provide the syntax of that command. These changes would provide great semantic value to the automaticly generated CLI documentation. I could also see having .long_help replaced by a mechanism that reads it from disk at runtime with a rudimentary Markdown/Doxygen filter so that we can use the same text that is used in the published documentation. Change-Id: I80d6fe349b47dce649fa77d21ffec0ddb45c7bbf Signed-off-by: Chris Luke <chrisy@flirble.org>
2016-07-25 16:38:11 -04:00
# Look for search/replace patterns
for p in replace_patterns:
str = p[0].sub(p[1], str)
VPP-221 CLI auto-documentation infrastructure As a step before Doxygen, extract CLI-related struct initializers from the code and parse that into a summary of the CLI commands available with the provided help text, such as it is. At the moment this only renders this into an indexed Markdown file that Doxygen then picks up but later we can use this information to enrich the existing VLIB_CLI_COMMAND macro documentor as well as provide runtime documentation to VPP that is stored on disk outside the binary image. Additionally support a comment block immediately prior to VLIB_CLI_COMMAND CLI command definitions in the form /*? ... ?*/ that can be used to include long-form documentation without having it compiled into VPP. Examples of documenting CLI commands can be found in vlib/vlib/unix/cli.c which, whilst not perfect, should provide a starting point. Screen captures of sample output can be seen at https://chrisy.flirble.org/vpp/doxy-cli-example.png and https://chrisy.flirble.org/vpp/doxy-cli-index.png . Next, shift the Doxygen root makefile targets to their own Makefile. The primary reason for this is that the siphon targets do dependency tracking which means it needs to generate those dependencies whenever make is run; that is pointless if we're not going to generate any documentation. This includes the package dependencies since they since they sometimes unnecessarily interfere with the code build in some cases at the moment; later we will look to building a Python venv to host the Python modules we use. One final remark: In future we may consider deprecating .long_help in the VLIB_CLI_COMMAND structure entirely but add perhaps .usage_help. .short_help would be reserved for a summary of the command function and .usage_help provide the syntax of that command. These changes would provide great semantic value to the automaticly generated CLI documentation. I could also see having .long_help replaced by a mechanism that reads it from disk at runtime with a rudimentary Markdown/Doxygen filter so that we can use the same text that is used in the published documentation. Change-Id: I80d6fe349b47dce649fa77d21ffec0ddb45c7bbf Signed-off-by: Chris Luke <chrisy@flirble.org>
2016-07-25 16:38:11 -04:00
sys.stdout.write(str+"\n")
VPP-221 CLI auto-documentation infrastructure As a step before Doxygen, extract CLI-related struct initializers from the code and parse that into a summary of the CLI commands available with the provided help text, such as it is. At the moment this only renders this into an indexed Markdown file that Doxygen then picks up but later we can use this information to enrich the existing VLIB_CLI_COMMAND macro documentor as well as provide runtime documentation to VPP that is stored on disk outside the binary image. Additionally support a comment block immediately prior to VLIB_CLI_COMMAND CLI command definitions in the form /*? ... ?*/ that can be used to include long-form documentation without having it compiled into VPP. Examples of documenting CLI commands can be found in vlib/vlib/unix/cli.c which, whilst not perfect, should provide a starting point. Screen captures of sample output can be seen at https://chrisy.flirble.org/vpp/doxy-cli-example.png and https://chrisy.flirble.org/vpp/doxy-cli-index.png . Next, shift the Doxygen root makefile targets to their own Makefile. The primary reason for this is that the siphon targets do dependency tracking which means it needs to generate those dependencies whenever make is run; that is pointless if we're not going to generate any documentation. This includes the package dependencies since they since they sometimes unnecessarily interfere with the code build in some cases at the moment; later we will look to building a Python venv to host the Python modules we use. One final remark: In future we may consider deprecating .long_help in the VLIB_CLI_COMMAND structure entirely but add perhaps .usage_help. .short_help would be reserved for a summary of the command function and .usage_help provide the syntax of that command. These changes would provide great semantic value to the automaticly generated CLI documentation. I could also see having .long_help replaced by a mechanism that reads it from disk at runtime with a rudimentary Markdown/Doxygen filter so that we can use the same text that is used in the published documentation. Change-Id: I80d6fe349b47dce649fa77d21ffec0ddb45c7bbf Signed-off-by: Chris Luke <chrisy@flirble.org>
2016-07-25 16:38:11 -04:00
# All done