kc3-lang/angle/src/compiler/preprocessor/Tokenizer.l

src/compiler/preprocessor/Tokenizer.l

Log

Author Commit Date CI Message
Geoff Lang 197d5294 2018-04-25T14:29:00 Wrap all preprocessor code in the angle namespace. BUG=836820 BUG=801364 Change-Id: I08b6a2f9f12b689e09df6efd916c313e71e8a051 Reviewed-on: https://chromium-review.googlesource.com/1028581 Commit-Queue: Geoff Lang <geofflang@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
U-GOOGLE\courtneygo bab086a7 2018-03-30T16:08:42 Fix Tokenizer issue Updated Tokenizer running generate_parser.sh on Windows. Test: make Change-Id: I128a70a61907bb4ce3b0a8031667bda4197b6a2d Reviewed-on: https://chromium-review.googlesource.com/988802 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Yuly Novikov <ynovikov@chromium.org> Commit-Queue: Courtney Goeltzenleuchter <courtneygo@google.com>
Nico Weber 41b072b6 2018-02-09T10:01:32 Prepare for -Wimplicit-fallthrough in ANGLE. Disable the warning for flex-generated output, which contains lots of intentional fallthrough. Fixes a bug where GL_SAMPLE_ALPHA_TO_ONE_EXT would fall through to GL_COVERAGE_MODULATION_CHROMIUM and hence behave like that. Fixes a bug in the D3D9 state management where invalidating DIRTY_BIT_POLYGON_OFFSET would also invalidate the stencil bits. One somewhat common incorrect pattern in ANGLE is nested switch statements that look like so: switch (a) { case a1: switch (b) { case b1: ... break; } case a2: ... } The assumption here seems to be that the breakk exits the outer case (here a1), while it in fact only exits the inner switch, so that we fall through to a2. In most places, I fixed this by adding an explicit `break` after the inner switch. This fixes a bug wher GL_PATH_JOIN_STYLE_CHROMIUM would fall through to GL_PATH_MITER_LIMIT_CHROMIUM in validation (but since the join style enum is always > 0, this happened to not have an effect in practice). This also fixes 87 bugs in GetLoadFunctionsMap() where invalid values would previously return an unrelated function map instead of the empty load function map. Bug: chromium:810767 Change-Id: Ib51388c73fbfc229160e2c10f8fb9364cc7c996c Reviewed-on: https://chromium-review.googlesource.com/911529 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Corentin Wallez d78e33a8 2017-10-30T12:33:52 preprocessor: Fix lineno overflow on line continuations BUG=chromium:774807 Change-Id: I4b3fbee31683f411810080572cfff0f8307b93bf Reviewed-on: https://chromium-review.googlesource.com/744183 Commit-Queue: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Corentin Wallez dc0fa46a 2017-02-01T14:44:43 preprocessor: Check for line number overflow Also remove dead code in Tokenizer.l BUG=chromium:668842 Change-Id: Ice18313a64f0bb2242299993bfaa882a6578ad54 Reviewed-on: https://chromium-review.googlesource.com/435042 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Corentin Wallez 054f7ed0 2016-09-20T17:15:59 preprocessor: Miscellaneous cleanups - Use full header paths in includes - Use ASSERT instead of assert - Use angle::NonCopyable instead of PP_DISALLOW_COPY_AND_ASSIGN - Use range-for in a couple places - Remove pp_utils.h BUG=angleproject:1522 Change-Id: If107fef89e8465bca65cf664926d1051c5d1e232 Reviewed-on: https://chromium-review.googlesource.com/387212 Commit-Queue: Corentin Wallez <cwallez@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Olli Etuaho 26e355b8 2015-08-14T14:16:19 Add full support for line continuation in the preprocessor Re-landing earlier change with constant signedness fixed (was causing build issues on Linux). Line continuation in ESSL 3.00 needs to be processed before tokenization, since tokens can span the line continuation. On the other hand, ANGLE's tokenizer keeps track of line numbers, and whenever a line continuation appears the line number still needs to be incremented by one, just like on a regular newline. That's why line continuation is now implemented as follows: when the shader strings are concatenated in Input, they are also checked for line continuation. Whenever line continuation is encountered, the string is cut before that point. When the tokenizer asks for more input, the string starting from the character after line continuation is passed to it, and the line number is incremented from Input. This way the tokenizer can parse tokens that span multiple lines - it never sees the line continuation - but still keeps track of the line number correctly. Relevant spec is in ESSL 3.00 section 3.2 "Source strings". Support for line continuation also applies to ESSL 1.00. ESSL 3.00 spec section 1.5 says that line continuation support is mandated when an ESSL 1.00 shader is used with the OpenGL ES 3.0 API, and is optional when ESSL 1.00 is used with the OpenGL ES 2.0 API. TEST=dEQP-GLES3.functional.shaders.preprocessor.line_continuation.* (all pass), angle_unittests BUG=angleproject:1125 Change-Id: Ic086aacac53cd75bf93c0fda782416501d2f842b Reviewed-on: https://chromium-review.googlesource.com/294200 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Olli Etuaho <oetuaho@nvidia.com>
Jamie Madill 3b040eb8 2015-08-17T16:51:33 Revert "Add full support for line continuation in the preprocessor" Warning in the Linux/Mac builders: In file included from ../../third_party/angle/src/tests/preprocessor_tests/input_test.cpp:7: In file included from ../../third_party/angle/src/tests/preprocessor_tests/PreprocessorTest.h:7: ../../testing/gtest/include/gtest/gtest.h:1392:16: error: comparison of integers of different signs: 'const int' and 'const unsigned long' [-Werror,-Wsign-compare] if (expected == actual) { ~~~~~~~~ ^ ~~~~~~ ../../testing/gtest/include/gtest/gtest.h:1422:12: note: in instantiation of function template specialization 'testing::internal::CmpHelperEQ<int, unsigned long>' requested here return CmpHelperEQ(expected_expression, actual_expression, expected, ^ ../../third_party/angle/src/tests/preprocessor_tests/input_test.cpp:171:5: note: in instantiation of function template specialization 'testing::internal::EqHelper<false>::Compare<int, unsigned long>' requested here EXPECT_EQ(3, input.read(buf, maxSize, &lineNo)); ^ BUG=angleproject:1125 This reverts commit c1157d1963170c7411eb6c32e2b2fbce02c5a170. Change-Id: Ic6fa286d190b006cccc5154d86e21ecc03175763 Reviewed-on: https://chromium-review.googlesource.com/294080 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Olli Etuaho c1157d19 2015-08-14T14:16:19 Add full support for line continuation in the preprocessor Line continuation in ESSL 3.00 needs to be processed before tokenization, since tokens can span the line continuation. On the other hand, ANGLE's tokenizer keeps track of line numbers, and whenever a line continuation appears the line number still needs to be incremented by one, just like on a regular newline. That's why line continuation is now implemented as follows: when the shader strings are concatenated in Input, they are also checked for line continuation. Whenever line continuation is encountered, the string is cut before that point. When the tokenizer asks for more input, the string starting from the character after line continuation is passed to it, and the line number is incremented from Input. This way the tokenizer can parse tokens that span multiple lines - it never sees the line continuation - but still keeps track of the line number correctly. Relevant spec is in ESSL 3.00 section 3.2 "Source strings". Support for line continuation also applies to ESSL 1.00. ESSL 3.00 spec section 1.5 says that line continuation support is mandated when an ESSL 1.00 shader is used with the OpenGL ES 3.0 API, and is optional when ESSL 1.00 is used with the OpenGL ES 2.0 API. TEST=dEQP-GLES3.functional.shaders.preprocessor.line_continuation.* (all pass), angle_unittests BUG=angleproject:1125 Change-Id: I1c1de49602e7cd755d6072c3c0aa5524cd0313b4 Reviewed-on: https://chromium-review.googlesource.com/293721 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Olli Etuaho <oetuaho@nvidia.com>
Scott Graham a897542f 2015-05-01T11:09:12 vs2015: disable warning in generated code, fix another Macro redefinition is for INT8_MIN, etc. which are generated by flex. VS doesn't define __STDC_VERSION__ as >= C99 because it's still only partial support in VS2015. Fix a float->int conversion narrowing warning. Change-Id: I5232eb23426eaf584218137c068e14d74119a1ef Reviewed-on: https://chromium-review.googlesource.com/268821 Reviewed-by: Zhenyao Mo <zmo@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Scott Graham <scottmg@chromium.org>
Jamie Madill 2dc8bf8b 2015-04-30T15:56:52 translator: Fix C++11-deprecated register usage. Define 'register' as an empty string if we're in a modern version of C++. BUG=255186 BUG=angleproject:463 Change-Id: Ied044fb87a9b05c91cb419c54295c39f0f0ab776 Reviewed-on: https://chromium-review.googlesource.com/268512 Reviewed-by: Geoff Lang <geofflang@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Jamie Madill b96687df 2015-04-30T15:56:51 translator: Regenerate with Bison 3.0.4. BUG=angleproject:463 Change-Id: If89a29de8fb006e8e3b0483ac5df21c8833ce974 Reviewed-on: https://chromium-review.googlesource.com/268511 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Minmin Gong 794e0009 2015-04-07T18:31:54 Fix and enable warning C4244 (Conversion from 'type1' to 'type2', possible loss of data) Change-Id: Id0e06d7d6600344d858f00dabc219d79289bbc82 Reviewed-on: https://chromium-review.googlesource.com/265020 Tested-by: Minmin Gong <mgong@microsoft.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Geoff Lang <geofflang@chromium.org>
Jamie Madill b3584fb4 2015-04-09T17:34:21 Revert "Fix and enable warning C4244 (Conversion from 'type1' to 'type2', possible loss of data)" Causing a build failure on Mac/Clang: ./Tokenizer.cpp:551:7: error: extra tokens at end of #else directive [-Werror,-Wextra-tokens] #else if defined(_MSC_VER) http://build.chromium.org/p/chromium.gpu.fyi/builders/GPU%20Mac%20Builder/builds/29136 This reverts commit 3b26e231d99154814eb428f75a67bbe7a21adadc. Change-Id: I2d11ddcc18130d908fd2ec3d6f5ab890cfccd5e7 Reviewed-on: https://chromium-review.googlesource.com/264983 Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Minmin Gong 3b26e231 2015-04-07T18:31:54 Fix and enable warning C4244 (Conversion from 'type1' to 'type2', possible loss of data) Change-Id: I73d9a2b9ad16f032be974b9c819de0dc1247c2ea Reviewed-on: https://chromium-review.googlesource.com/264533 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Jamie Madill <jmadill@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 185de884 2014-12-22T15:17:52 Update ANGLE's translator to Bison 3. BUG=angle:462 Change-Id: I2c1c18027dee1c3b4efb87374caaadbf58367841 Reviewed-on: https://chromium-review.googlesource.com/236930 Tested-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Kenneth Russell <kbr@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org>
Zhenyao Mo d526f989 2014-05-13T14:51:19 Fix code style violation in compiler/preprocessor BUG=angle:650 TEST=no behavior change Change-Id: Ib52f15f6471fc7897b66d11baee11216cf08158a Reviewed-on: https://chromium-review.googlesource.com/199591 Reviewed-by: Jamie Madill <jmadill@chromium.org> Reviewed-by: Zhenyao Mo <zmo@chromium.org> Tested-by: Zhenyao Mo <zmo@chromium.org>
Jamie Madill 5508f39d 2014-02-20T13:31:36 Fix breaking the build with missing GetGlobalMaxTokenSize. Because the preprocessor is used independently from the compiler, we need a way to track max token size when we don't have access to the parse context with the current spec. BUG=angle:550 Change-Id: Idf5035ec2c001ee75f264151ab3c4e92f3cd44d7 Reviewed-on: https://chromium-review.googlesource.com/187140 Reviewed-by: Geoff Lang <geofflang@chromium.org> Reviewed-by: Shannon Woods <shannonwoods@chromium.org> Reviewed-by: Nicolas Capens <nicolascapens@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill 88f6e946 2014-02-19T10:27:53 Proper support for token max size in WebGL+ES3. WebGL specifies a maximum token size of 256 characters, while ES3 specifies 1024 characters. We can determine the proper max size to support from the spec. BUG=angle:550 Change-Id: I6aeabe8af3b6184a27b456248ce2f84f361b76e4 Reviewed-on: https://chromium-review.googlesource.com/186973 Reviewed-by: Shannon Woods <shannonwoods@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
Jamie Madill c9f140d8 2014-02-18T15:27:21 Add preprocess bison files to the gyp files. We had accidentally left out the y and l files from our generation scripts, causing us to miss several instances of updated enum names. BUG=angle:550 Change-Id: I8790742fbaab5435e4c0db4f61c3e8194a231550 Reviewed-on: https://chromium-review.googlesource.com/186972 Reviewed-by: Nicolas Capens <nicolascapens@chromium.org> Tested-by: Jamie Madill <jmadill@chromium.org>
shannonwoods@chromium.org 2a5436ff 2013-05-30T00:21:41 Added preprocessor support for parsing the float suffix. TRAC #23185 Signed-off-by: Geoff Lang Signed-off-by: Shannon Woods Author: Nicolas Capens git-svn-id: https://angleproject.googlecode.com/svn/branches/es3proto@2413 736b8ea6-26fd-11df-bfd4-992fa37f6226
shannonwoods@chromium.org 3f83e29c 2013-05-30T00:21:34 Added preprocessor support for parsing the unsigned integer suffix. TRAC #23185 Signed-off-by: Geoff Lang Signed-off-by: Shannon Woods Author: Nicolas Capens git-svn-id: https://angleproject.googlecode.com/svn/branches/es3proto@2412 736b8ea6-26fd-11df-bfd4-992fa37f6226
shannonwoods@chromium.org c8100b85 2013-05-30T00:20:34 Add support for unsigned integer literals in the shading language. TRAC #23080 Signed-off-by: Nicolas Capens Signed-off-by: Shannon Woods Author: Jamie Madill git-svn-id: https://angleproject.googlecode.com/svn/branches/es3proto@2405 736b8ea6-26fd-11df-bfd4-992fa37f6226
shannonwoods@chromium.org 8ddaf5c0 2013-05-30T00:13:08 Increase the maximum token length in the preprocessor to be 1024, to comply with the GLSL ES 3 specification. TRAC #23077 Signed-off-by: Shannon Woods Signed-off-by: Geoff Lang Author: Jamie Madill git-svn-id: https://angleproject.googlecode.com/svn/branches/es3proto@2349 736b8ea6-26fd-11df-bfd4-992fa37f6226
shannon.woods%transgaming.com@gtempaccount.com bcde56f7 2013-04-13T03:32:12 Added support for line continuations. TRAC #22713 Signed-off-by: Geoff Lang Signed-off-by: Shannon Woods Author: Nicolas Capens git-svn-id: https://angleproject.googlecode.com/svn/branches/es3proto@2086 736b8ea6-26fd-11df-bfd4-992fa37f6226
shannon.woods@transgaming.com 8e02e356 2013-02-28T23:20:08 Fix ANGLE patching/building issues. 1) Remove an empty file: this causes patch apply failures when updating WebKit side ANGLE. 2) Fix a 64-to-32 conversion issue. 3) Append a change to Tokenizer.l that's left out in a previous CL. (cpp is already updated). Review URL: https://codereview.appspot.com/7378051 git-svn-id: https://angleproject.googlecode.com/svn/branches/dx11proto@1965 736b8ea6-26fd-11df-bfd4-992fa37f6226
shannon.woods@transgaming.com d64b3dab 2013-02-28T23:19:26 Fixed 64-bit integer truncation issues in shader translator. This is an incompatible API change, but one which is necessary in order to improve correctness of the code. The API version in ShaderLang.h is updated and, unfortunately, the define renamed to something less ambiguous due to conflicts on some Android buildbots. Temporary patches in Chromium and WebKit will be landed separately to support this upgrade. BUG=403,404,405,406,407,408,409 Review URL: https://codereview.appspot.com/7300058 Conflicts: include/GLSLANG/ShaderLang.h git-svn-id: https://angleproject.googlecode.com/svn/branches/dx11proto@1960 736b8ea6-26fd-11df-bfd4-992fa37f6226
daniel@transgaming.com b3077d08 2013-01-11T04:12:09 Upstream various build fixes from WebKit to ANGLE to make updating ANGLE in WebKit easier. a) http://trac.webkit.org/changeset/127747 b) http://trac.webkit.org/changeset/128539 c) http://trac.webkit.org/changeset/122870 - Specifically, items #3 and #4 in this changeset's commit message. Review URL: https://codereview.appspot.com/7040045 Author: maxvujovic@gmail.com ------ Upodate preprocessor.vcxproj to reflect changes in r1640. Review URL: https://codereview.appspot.com/7061044 git-svn-id: https://angleproject.googlecode.com/svn/branches/dx11proto@1703 736b8ea6-26fd-11df-bfd4-992fa37f6226
daniel@transgaming.com b401a92b 2012-10-26T18:58:24 Move the new preprocessor out of the 'new' directory. TRAC #21966 Signed-off-by: Daniel Koch Author: Nicolas Capens git-svn-id: https://angleproject.googlecode.com/svn/trunk@1326 736b8ea6-26fd-11df-bfd4-992fa37f6226
kmx.io     mail     discord kmxgit v0.4.0