kc3-lang/angle/src/compiler/preprocessor/new/Tokenizer.cpp

Branch :


Log

Author Commit Date CI Message
c022c3af 2012-07-09 15:56:42 WebGL spec specifies maximum length of all types of tokens - not just identifier tokens. And it also means preprocessing-tokens, not compiler tokens. Note that this implies that non-compliant tokens even inside excluded #if blocks will trigger error. TODO: This behavior should be implemented as a preprocessor option, so that a GLES2 compiler can choose to disable it. Review URL: https://codereview.appspot.com/6355066 git-svn-id: https://angleproject.googlecode.com/svn/trunk@1193 736b8ea6-26fd-11df-bfd4-992fa37f6226
5b6a68e0 2012-06-28 20:29:13 Replaced pp::Token::value with pp::Token::text. The term value will be used for a function which will convert text to integer/float constant. git-svn-id: https://angleproject.googlecode.com/svn/trunk@1175 736b8ea6-26fd-11df-bfd4-992fa37f6226
c6b3b3c7 2012-06-27 22:49:39 Fix the compiler warnings on WebKit ports when updating ANGLE in WebKit. Remove the varargs used for extra info formatting in the error() and warning() methods of ParseHelper. Use std::stringstream for formatting instead. Review URL: http://codereview.appspot.com/6310067/ git-svn-id: https://angleproject.googlecode.com/svn/trunk@1170 736b8ea6-26fd-11df-bfd4-992fa37f6226
432d6fc4 2012-06-27 22:13:21 Introduced preprocessing token types. This fixes a bug where invalid tokens inside excluded conditional block may report diagnostics. Now we let the invalid tokens to bubble through the preprocessor so that they have chance to be skipped. Review URL: https://codereview.appspot.com/6356045 git-svn-id: https://angleproject.googlecode.com/svn/trunk@1169 736b8ea6-26fd-11df-bfd4-992fa37f6226
46aa13d8 2012-06-15 15:40:27 Implemented line directive. Review URL: https://codereview.appspot.com/6307083 git-svn-id: https://angleproject.googlecode.com/svn/trunk@1149 736b8ea6-26fd-11df-bfd4-992fa37f6226
7fc38ddd 2012-06-14 18:23:23 Implemented macro expansion. Review URL: https://codereview.appspot.com/6303052 git-svn-id: https://angleproject.googlecode.com/svn/trunk@1148 736b8ea6-26fd-11df-bfd4-992fa37f6226
19d7aa60 2012-05-31 17:34:05 Fixed the location of EOF token. Added three new location tests for EOF. git-svn-id: https://angleproject.googlecode.com/svn/trunk@1125 736b8ea6-26fd-11df-bfd4-992fa37f6226
07d921de 2012-05-22 20:22:08 Token::value now contains text for all types of tokens - not just CONST_INT, CONST_FLOAT, and IDENTIFIER. This makes it easier to report diagnostics and various preprocessor directives. git-svn-id: https://angleproject.googlecode.com/svn/trunk@1093 736b8ea6-26fd-11df-bfd4-992fa37f6226
2c958eef 2012-05-17 20:35:42 Moved error-handling to a separate class - Diagnostics. We were earlier returning errors as tokens which did not work very well when error occured while parsing a preprocessor directive. Now all returned tokens are valid. Errors are reported via an abstract Diagnostics interface. Updated unit-tests with the new scheme. Review URL: https://codereview.appspot.com/6203089 git-svn-id: https://angleproject.googlecode.com/svn/trunk@1087 736b8ea6-26fd-11df-bfd4-992fa37f6226
04d7d22b 2012-05-16 19:24:07 Reafactored Lexer class to allow chaining. The full chain when parsing #if directive looks like this: Preprocessor -> MacroExpander -> DirectiveHandler -> MacroExpander -> DefinedFilter -> Tokenizer. This chain dynamically changes depending on the current context. Also added an incomplete implementation of #if handling and ExpressionParser to illustrate how this design is supposed to work. Review URL: https://codereview.appspot.com/6203060 git-svn-id: https://angleproject.googlecode.com/svn/trunk@1084 736b8ea6-26fd-11df-bfd4-992fa37f6226