src/program_serialize_data_version.py


Log

Author Commit Date CI Message
Yuxin Hu 82826be0 2023-01-06T11:20:14 Reland "Make ANGLE program version only dependent on data that matters" This is a reland of commit 61728827d2e5ecce685578bc54bb2c744b65fc9a Changes made on top of previous commit 1. In src/program_serialize_data_version.py, fix the script error on Windows: anglebug.com/7918. 2. In BUILD.gn, replace the 'write_file' with 'response_file_content'. See https://gn.googlesource.com/gn/+/main/docs/reference.md#var_response_file_contents 3. In scripts/generate_android.bp.py, add code to handle the special GN build flag {{response_file_name}}. The code writes the list defined in srcs (identified by $(in) in Android blueprint) into a temp file named 'gn_response_file', and replaces the {{response_file_name}} with 'gn_response_file'. Original change's description > Changes made on top of original commit > 1. Enable execution permission on python script > program_serialize_data_version.py > 2. Remove unused list in libGLESv2.gni > 3. In angle/BUILD.gn, change file path from > "relative to angle_root", to "relative to root_build_dir", > so that inside the script program_serialize_data_version.py, > we don't have to find the absolute path of the code files for > hashing. > Original change's description > > This change introduces a new variable ANGLE_PROGRAM_VERSION > > to track the version of ANGLE source files that affect shader > > program serialization/deserialization. This change include more > > source files than necessary, to serve the purpose of a conservative > > jumping off point. We will narrow down the list of files for > > ANGLE_PROGRAM_VERSION hash generation in the future. > > Add a new script program_serialize_data_version.py that will > > be triggered during the build when the related source files changed. > > The script will generate a hash and the hash size from the related > > source files. In program serialization/deserialization and cache > > key generation, we will use this hash value instead of the entire > > ANGLE git hash. When the hash value changed, we know that the > > related source files changed, and we should invalidate the program > > cache and re-generate the blob cache / program binary. > > Bug: angleproject:4981 > > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > > Reviewed-by: Cody Northrop <cnorthrop@google.com> > > Reviewed-by: Jamie Madill <jmadill@chromium.org> > > Commit-Queue: Yuxin Hu <yuxinhu@google.com> > Bug: angleproject:4981 > Change-Id: Iaa9eb0ab33439197bc30d03064fc245ea7ef1ea8 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4113445 > Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: Ib5bba199be6d08a1e19807026df0e7b747dbc8a9 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4144078 Reviewed-by: Roman Lavrov <romanl@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com>
Yuxin Hu c4b95929 2023-01-05T19:59:07 Revert "Reland "Make ANGLE program version only dependent on data that matters"" This reverts commit 61728827d2e5ecce685578bc54bb2c744b65fc9a. Reason for revert: it breaks aosp build: https://android-review.git.corp.google.com/c/platform/external/angle/+/2374971. Build error: `FileNotFoundError: [Errno 2] No such file or directory: 'angle_code_affecting_program_serialize'` Original change's description: > Reland "Make ANGLE program version only dependent on data that matters" > > This is a reland of commit c303758fbc8d23266be578fae246e5687656b4c6 > > Changes made on top of original commit > 1. Enable execution permission on python script > program_serialize_data_version.py > 2. Remove unused list in libGLESv2.gni > 3. In angle/BUILD.gn, change file path from > "relative to angle_root", to "relative to root_build_dir", > so that inside the script program_serialize_data_version.py, > we don't have to find the absolute path of the code files for > hashing. > > Original change's description > > > This change introduces a new variable ANGLE_PROGRAM_VERSION > > to track the version of ANGLE source files that affect shader > > program serialization/deserialization. This change include more > > source files than necessary, to serve the purpose of a conservative > > jumping off point. We will narrow down the list of files for > > ANGLE_PROGRAM_VERSION hash generation in the future. > > > Add a new script program_serialize_data_version.py that will > > be triggered during the build when the related source files changed. > > The script will generate a hash and the hash size from the related > > source files. In program serialization/deserialization and cache > > key generation, we will use this hash value instead of the entire > > ANGLE git hash. When the hash value changed, we know that the > > related source files changed, and we should invalidate the program > > cache and re-generate the blob cache / program binary. > > > Bug: angleproject:4981 > > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > > Reviewed-by: Cody Northrop <cnorthrop@google.com> > > Reviewed-by: Jamie Madill <jmadill@chromium.org> > > Commit-Queue: Yuxin Hu <yuxinhu@google.com> > > Bug: angleproject:4981 > Change-Id: Iaa9eb0ab33439197bc30d03064fc245ea7ef1ea8 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4113445 > Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: If7d3779c8f40dafa9a9c057f669ab53d0494526e Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4140138 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com>
Yuxin Hu 61728827 2022-12-14T15:29:15 Reland "Make ANGLE program version only dependent on data that matters" This is a reland of commit c303758fbc8d23266be578fae246e5687656b4c6 Changes made on top of original commit 1. Enable execution permission on python script program_serialize_data_version.py 2. Remove unused list in libGLESv2.gni 3. In angle/BUILD.gn, change file path from "relative to angle_root", to "relative to root_build_dir", so that inside the script program_serialize_data_version.py, we don't have to find the absolute path of the code files for hashing. Original change's description > This change introduces a new variable ANGLE_PROGRAM_VERSION > to track the version of ANGLE source files that affect shader > program serialization/deserialization. This change include more > source files than necessary, to serve the purpose of a conservative > jumping off point. We will narrow down the list of files for > ANGLE_PROGRAM_VERSION hash generation in the future. > Add a new script program_serialize_data_version.py that will > be triggered during the build when the related source files changed. > The script will generate a hash and the hash size from the related > source files. In program serialization/deserialization and cache > key generation, we will use this hash value instead of the entire > ANGLE git hash. When the hash value changed, we know that the > related source files changed, and we should invalidate the program > cache and re-generate the blob cache / program binary. > Bug: angleproject:4981 > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: Iaa9eb0ab33439197bc30d03064fc245ea7ef1ea8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4113445 Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Reviewed-by: Cody Northrop <cnorthrop@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com>
Yuxin Hu 25ce2e23 2022-12-16T18:16:32 Revert "Make ANGLE program version only dependent on data that matters" This reverts commit c303758fbc8d23266be578fae246e5687656b4c6. Reason for revert: This is causing the Android roller fail: https://crrev.com/c/4072215/comments/6918d7f6_8100d66b?tab=comments Original change's description: > Make ANGLE program version only dependent on data that matters > > This change introduces a new variable ANGLE_PROGRAM_VERSION > to track the version of ANGLE source files that affect shader > program serialization/deserialization. This change include more > source files than necessary, to serve the purpose of a conservative > jumping off point. We will narrow down the list of files for > ANGLE_PROGRAM_VERSION hash generation in the future. > > Add a new script program_serialize_data_version.py that will > be triggered during the build when the related source files changed. > The script will generate a hash and the hash size from the related > source files. In program serialization/deserialization and cache > key generation, we will use this hash value instead of the entire > ANGLE git hash. When the hash value changed, we know that the > related source files changed, and we should invalidate the program > cache and re-generate the blob cache / program binary. > > Bug: angleproject:4981 > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> > Reviewed-by: Cody Northrop <cnorthrop@google.com> > Reviewed-by: Jamie Madill <jmadill@chromium.org> > Commit-Queue: Yuxin Hu <yuxinhu@google.com> Bug: angleproject:4981 Change-Id: Ib235d85c79ed54110696ac5413571934890fdc98 No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4114274 Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com> Commit-Queue: Yuxin Hu <yuxinhu@google.com>
Yuxin Hu c303758f 2022-12-14T15:29:15 Make ANGLE program version only dependent on data that matters This change introduces a new variable ANGLE_PROGRAM_VERSION to track the version of ANGLE source files that affect shader program serialization/deserialization. This change include more source files than necessary, to serve the purpose of a conservative jumping off point. We will narrow down the list of files for ANGLE_PROGRAM_VERSION hash generation in the future. Add a new script program_serialize_data_version.py that will be triggered during the build when the related source files changed. The script will generate a hash and the hash size from the related source files. In program serialization/deserialization and cache key generation, we will use this hash value instead of the entire ANGLE git hash. When the hash value changed, we know that the related source files changed, and we should invalidate the program cache and re-generate the blob cache / program binary. Bug: angleproject:4981 Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215 Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org> Reviewed-by: Cody Northrop <cnorthrop@google.com> Reviewed-by: Jamie Madill <jmadill@chromium.org> Commit-Queue: Yuxin Hu <yuxinhu@google.com>