kc3-lang/angle/src/program_serialize_data_version.py

• Show log

  Commit

• Hash : 61728827
  Author :
  Date : 2022-12-14T15:29:15
  

  Reland "Make ANGLE program version only dependent on data that matters"
  
  This is a reland of commit c303758fbc8d23266be578fae246e5687656b4c6
  
  Changes made on top of original commit
  1. Enable execution permission on python script
  program_serialize_data_version.py
  2. Remove unused list in libGLESv2.gni
  3. In angle/BUILD.gn, change file path from
  "relative to angle_root", to "relative to root_build_dir",
  so that inside the script program_serialize_data_version.py,
  we don't have to find the absolute path of the code files for
  hashing.
  
  Original change's description
  
  > This change introduces a new variable ANGLE_PROGRAM_VERSION
  > to track the version of ANGLE source files that affect shader
  > program serialization/deserialization. This change include more
  > source files than necessary, to serve the purpose of a conservative
  > jumping off point. We will narrow down the list of files for
  > ANGLE_PROGRAM_VERSION hash generation in the future.
  
  > Add a new script program_serialize_data_version.py that will
  > be triggered during the build when the related source files changed.
  > The script will generate a hash and the hash size from the related
  > source files. In program serialization/deserialization and cache
  > key generation, we will use this hash value instead of the entire
  > ANGLE git hash. When the hash value changed, we know that the
  > related source files changed, and we should invalidate the program
  > cache and re-generate the blob cache / program binary.
  
  > Bug: angleproject:4981
  > Change-Id: I2fb609416738d459d3289190c232c2d797ba58e3
  > Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4072215
  > Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
  > Reviewed-by: Cody Northrop <cnorthrop@google.com>
  > Reviewed-by: Jamie Madill <jmadill@chromium.org>
  > Commit-Queue: Yuxin Hu <yuxinhu@google.com>
  
  Bug: angleproject:4981
  Change-Id: Iaa9eb0ab33439197bc30d03064fc245ea7ef1ea8
  Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/4113445
  Reviewed-by: Amirali Abdolrashidi <abdolrashidi@google.com>
  Reviewed-by: Cody Northrop <cnorthrop@google.com>
  Commit-Queue: Yuxin Hu <yuxinhu@google.com>
  

Download

• src/program_serialize_data_version.py

• #!/usr/bin/env python3
  #  Copyright 2022 The ANGLE Project Authors. All rights reserved.
  #  Use of this source code is governed by a BSD-style license that can be
  #  found in the LICENSE file.
  
  # Generate ANGLEShaderProgramVersion.h with hash of files affecting data
  # used in serializing/deserializing shader programs.
  
  import hashlib
  import argparse
  
  
  def GenerateHashOfAffectedFiles(angle_code_files):
      hash_md5 = hashlib.md5()
      for file in angle_code_files:
          assert (file != "")
          with open(file, 'r') as f:
              for chunk in iter(lambda: f.read(4096), ""):
                  hash_md5.update(chunk.encode())
      return hash_md5.hexdigest(), hash_md5.digest_size
  
  
  parser = argparse.ArgumentParser(description='Generate the file ANGLEShaderProgramVersion.h')
  parser.add_argument(
      'output_file',
      help='path (relative to build directory) to output file name, stores ANGLE_PROGRAM_VERSION and ANGLE_PROGRAM_VERSION_HASH_SIZE'
  )
  parser.add_argument(
      'input_file',
      help='path (relative to build directory) to input file name, the input file stores a list of program files that ANGLE_PROGRAM_VERSION hashes over'
  )
  args = parser.parse_args()
  output_file = args.output_file
  input_file = args.input_file
  with open(input_file, "r") as input_files:
      angle_code_files = input_files.read().split('\n')
  #remove the last empty item because of the '\n' in input_file (file $root_gen_dir/angle_code_affecting_program_serialize)
  if (len(angle_code_files) > 0):
      assert (angle_code_files.pop() == "")
  angle_shader_program_version_hash_result = GenerateHashOfAffectedFiles(angle_code_files)
  hfile = open(output_file, 'w')
  hfile.write('#define ANGLE_PROGRAM_VERSION "%s"\n' % angle_shader_program_version_hash_result[0])
  hfile.write('#define ANGLE_PROGRAM_VERSION_HASH_SIZE %d\n' %
              angle_shader_program_version_hash_result[1])
  hfile.close()
  

Download