)]}' { "commit": "8c7a6feb8e3b3328995ea033ccbcf7f817b9b41c", "tree": "aa0a6c0091ebe5af70f6eb6ea1cb326e0b224257", "parents": [ "4270c62812016c71db82e4d780108bb3b7c0d539" ], "author": { "name": "Anuj Rawat", "email": "anuj.rawat@intel.com", "time": "Sat Apr 20 06:46:43 2019 +0000" }, "committer": { "name": "Anuj Rawat", "email": "anuj.rawat@intel.com", "time": "Sat Apr 20 06:46:43 2019 +0000" }, "message": "Adding lowlevel APIs for optimized RHS packet load in TensorFlow\n SpatialConvolution\n\nLow-level APIs are added in order to optimized packet load in gemm_pack_rhs\nin TensorFlow SpatialConvolution. The optimization is for scenario when a\npacket is split across 2 adjacent columns. In this case we read it as two\n\u0027partial\u0027 packets and then merge these into 1. Currently this only works for\nPacket16f (AVX512) and Packet8f (AVX2). We plan to add this for other\npacket types (such as Packet8d) also.\n\nThis optimization shows significant speedup in SpatialConvolution with\ncertain parameters. Some examples are below.\n\nBenchmark parameters are specified as:\nBatch size, Input dim, Depth, Num of filters, Filter dim\n\nSpeedup numbers are specified for number of threads 1, 2, 4, 8, 16.\n\nAVX512:\n\nParameters | Speedup (Num of threads: 1, 2, 4, 8, 16)\n----------------------------|------------------------------------------\n128, 24x24, 3, 64, 5x5 |2.18X, 2.13X, 1.73X, 1.64X, 1.66X\n128, 24x24, 1, 64, 8x8 |2.00X, 1.98X, 1.93X, 1.91X, 1.91X\n 32, 24x24, 3, 64, 5x5 |2.26X, 2.14X, 2.17X, 2.22X, 2.33X\n128, 24x24, 3, 64, 3x3 |1.51X, 1.45X, 1.45X, 1.67X, 1.57X\n 32, 14x14, 24, 64, 5x5 |1.21X, 1.19X, 1.16X, 1.70X, 1.17X\n128, 128x128, 3, 96, 11x11 |2.17X, 2.18X, 2.19X, 2.20X, 2.18X\n\nAVX2:\n\nParameters | Speedup (Num of threads: 1, 2, 4, 8, 16)\n----------------------------|------------------------------------------\n128, 24x24, 3, 64, 5x5 | 1.66X, 1.65X, 1.61X, 1.56X, 1.49X\n 32, 24x24, 3, 64, 5x5 | 1.71X, 1.63X, 1.77X, 1.58X, 1.68X\n128, 24x24, 1, 64, 5x5 | 1.44X, 1.40X, 1.38X, 1.37X, 1.33X\n128, 24x24, 3, 64, 3x3 | 1.68X, 1.63X, 1.58X, 1.56X, 1.62X\n128, 128x128, 3, 96, 11x11 | 1.36X, 1.36X, 1.37X, 1.37X, 1.37X\n\nIn the higher level benchmark cifar10, we observe a runtime improvement\nof around 6% for AVX512 on Intel Skylake server (8 cores).\n\nOn lower level PackRhs micro-benchmarks specified in TensorFlow\ntensorflow/core/kernels/eigen_spatial_convolutions_test.cc, we observe\nthe following runtime numbers:\n\nAVX512:\n\nParameters | Runtime without patch (ns) | Runtime with patch (ns) | Speedup\n---------------------------------------------------------------|----------------------------|-------------------------|---------\nBM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56) | 41350 | 15073 | 2.74X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56) | 7277 | 7341 | 0.99X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56) | 8675 | 8681 | 1.00X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56) | 24155 | 16079 | 1.50X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56) | 25052 | 17152 | 1.46X\nBM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) | 18269 | 18345 | 1.00X\nBM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) | 19468 | 19872 | 0.98X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432) | 156060 | 42432 | 3.68X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432) | 132701 | 36944 | 3.59X\n\nAVX2:\n\nParameters | Runtime without patch (ns) | Runtime with patch (ns) | Speedup\n---------------------------------------------------------------|----------------------------|-------------------------|---------\nBM_RHS_NAME(PackRhs, 128, 24, 24, 3, 64, 5, 5, 1, 1, 256, 56) | 26233 | 12393 | 2.12X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 1, 1, 256, 56) | 6091 | 6062 | 1.00X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 32, 64, 5, 5, 2, 2, 256, 56) | 7427 | 7408 | 1.00X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 1, 1, 256, 56) | 23453 | 20826 | 1.13X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 30, 64, 5, 5, 2, 2, 256, 56) | 23167 | 22091 | 1.09X\nBM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 1, 1, 256, 56) | 23422 | 23682 | 0.99X\nBM_RHS_NAME(PackRhs, 32, 256, 256, 4, 16, 8, 8, 2, 4, 256, 56) | 23165 | 23663 | 0.98X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 1, 1, 36, 432) | 72689 | 44969 | 1.62X\nBM_RHS_NAME(PackRhs, 32, 64, 64, 4, 16, 3, 3, 2, 2, 36, 432) | 61732 | 39779 | 1.55X\n\nAll benchmarks on Intel Skylake server with 8 cores.\n", "tree_diff": [ { "type": "modify", "old_id": "04a321b9fda6c6f1142b5a17d89f1199e6368b5d", "old_mode": 33188, "old_path": "Eigen/src/Core/GenericPacketMath.h", "new_id": "3bba0227886f9da245e00d379e9255b19cc44fe6", "new_mode": 33188, "new_path": "Eigen/src/Core/GenericPacketMath.h" }, { "type": "modify", "old_id": "5b8ff59bd965ffb349a51447dda7b8a50e3c8fea", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/AVX/Complex.h", "new_id": "3d229cd81751d0a341229d0ab2aba093ef5d0afd", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/AVX/Complex.h" }, { "type": "modify", "old_id": "f88e360241dda5f9bb1a001b42fad5a897301947", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/AVX/PacketMath.h", "new_id": "3f94f85eb15e28be6519a3ca0e0c35a18374507e", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/AVX/PacketMath.h" }, { "type": "modify", "old_id": "9a89dd01f9c29a87fc42f35fb0cfadfb72bafb27", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/AVX512/Complex.h", "new_id": "5ab2ffe774c50d523152fe0a7d2b52494505a6df", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/AVX512/Complex.h" }, { "type": "modify", "old_id": "60b723b08c24e7a74a004bb6a72f9b43a438e78a", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/AVX512/PacketMath.h", "new_id": "094309eda25b31666018ae958efa39a852602b9f", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/AVX512/PacketMath.h" }, { "type": "modify", "old_id": "0c9f8e05d9126dfe4c97e299ab1aa665940052aa", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/AltiVec/Complex.h", "new_id": "ebc3b2aeb208573fb0ec30a10983e5f26656eb8e", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/AltiVec/Complex.h" }, { "type": "modify", "old_id": "9535724ebe09b4f1b5cb5f6179d9b3b788bff5a7", "old_mode": 33261, "old_path": "Eigen/src/Core/arch/AltiVec/PacketMath.h", "new_id": "b5484e6aaceb4ee5f6faaf986bf8b364ecaabb3a", "new_mode": 33261, "new_path": "Eigen/src/Core/arch/AltiVec/PacketMath.h" }, { "type": "modify", "old_id": "cd4615a453def3c529e826f0ba9db885260db3f2", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/GPU/PacketMath.h", "new_id": "7fac0a5e16232a55867a16cd0705bee1f62ab927", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/GPU/PacketMath.h" }, { "type": "modify", "old_id": "cd518c7e409496409eeae639c559446b4535cf46", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/GPU/PacketMathHalf.h", "new_id": "2bee56f0fdee59f56edd726e0fb2734648515eb1", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/GPU/PacketMathHalf.h" }, { "type": "modify", "old_id": "fa64d35643a7021e504698fe4aa767942fbab74d", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/MSA/Complex.h", "new_id": "0ced061c5320e0550fd599c35a6b373211a97efd", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/MSA/Complex.h" }, { "type": "modify", "old_id": "a97156a84cd0a9c5c36917909a8994b683396763", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/MSA/PacketMath.h", "new_id": "f426d5b835ae55bffc906a9b6728ee9965e741ec", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/MSA/PacketMath.h" }, { "type": "modify", "old_id": "f6c5c211c02cf52794119fcfc270236dcdfcc0be", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/NEON/Complex.h", "new_id": "c17d0a03e9ad765b8808bcb9de3183f4938e9088", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/NEON/Complex.h" }, { "type": "modify", "old_id": "e8b3518492747d8fc9486fe93e65f7d25588fe1d", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/NEON/PacketMath.h", "new_id": "b8051cfbfbcd4d88a1fc0b36539bdb5b0a14f274", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/NEON/PacketMath.h" }, { "type": "modify", "old_id": "f39988eaca5b3494d0ee08351d738ea120c90269", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/SSE/Complex.h", "new_id": "7d89c323b6fdc6baf0227b1c0078c4271ce3d773", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/SSE/Complex.h" }, { "type": "modify", "old_id": "9c3750af0b57fab9e4ab559fa871776e2c02ea91", "old_mode": 33261, "old_path": "Eigen/src/Core/arch/SSE/PacketMath.h", "new_id": "04b63601b99e72e744a17904d684b5dda36896ad", "new_mode": 33261, "new_path": "Eigen/src/Core/arch/SSE/PacketMath.h" }, { "type": "modify", "old_id": "294cb101ab6d6cf3ae13d8470cd20e1507a8d0ea", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/SYCL/InteropHeaders.h", "new_id": "1afa63bb2d93563f6d139f629e47981d66bae7f0", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/SYCL/InteropHeaders.h" }, { "type": "modify", "old_id": "167c3ee4c9797bbb50be718086cae866ba9bf567", "old_mode": 33188, "old_path": "Eigen/src/Core/arch/ZVector/Complex.h", "new_id": "9fcbcb82ddf5919fbcce5a1aa8c586f5819cf5e3", "new_mode": 33188, "new_path": "Eigen/src/Core/arch/ZVector/Complex.h" }, { "type": "modify", "old_id": "c8e90f1a8f0b9f8b8506929e129df48967bd8126", "old_mode": 33261, "old_path": "Eigen/src/Core/arch/ZVector/PacketMath.h", "new_id": "74e0a13139c30459d7fca972e1a666c179b4847c", "new_mode": 33261, "new_path": "Eigen/src/Core/arch/ZVector/PacketMath.h" }, { "type": "modify", "old_id": "91c2e42e461c32cba0d2feb50489126ef7923c4a", "old_mode": 33188, "old_path": "Eigen/src/Core/util/XprHelper.h", "new_id": "ce019946bf724a301731948d41d9fb5e8bfb5681", "new_mode": 33188, "new_path": "Eigen/src/Core/util/XprHelper.h" }, { "type": "modify", "old_id": "4906f6eb050c5498eb09cdde3c39b0eb0d4e77cd", "old_mode": 33188, "old_path": "test/packetmath.cpp", "new_id": "200670b8c24af80228f30b1b0cf414bf5f60d413", "new_mode": 33188, "new_path": "test/packetmath.cpp" }, { "type": "modify", "old_id": "4ca6b3d8c625bc50f4821cd16a77413c331cc047", "old_mode": 33188, "old_path": "unsupported/Eigen/CXX11/src/Tensor/TensorEvaluator.h", "new_id": "05c684ff96be18de69201e9e8a3bf6c4a3fef3d2", "new_mode": 33188, "new_path": "unsupported/Eigen/CXX11/src/Tensor/TensorEvaluator.h" } ] }