TY - GEN
T1 - Sequential algorithms to split and merge ultra-high resolution 3D images
AU - Hayot-Sasson, Valérie
AU - Gao, Yongping
AU - Yan, Yuhong
AU - Glatard, Tristan
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2017/7/1
Y1 - 2017/7/1
N2 - Splitting and merging data is a requirement for many parallel or distributed processing operations. Naive algorithms to split and merge 3D blocks from ultra-high resolution images perform very poorly, as a result of seek times. In contrast, naive algorithms to split and merge 3D slabs perform optimally as seek time is significantly minimized. We introduce and analyze sequential algorithms (Clustered reads, Multiple reads, Clustered writes, and Multiple writes) that leverage memory buffering to address this issue. Clustered reads and Clustered writes, access image chunks only once, but they have to seek in the reconstructed image. Multiple reads and Multiple writes minimize seeks in the reconstructed image, but they access image chunks multiple times. Evaluation on a 3850×3025×3500 brain image shows that our algorithms perform similarly to the optimal configuration provided that enough memory is available. Additionally, Multiple reads supports on-the-fly compression of the merged image transparently but Clustered reads does not, due to its use of negative seeking. We conclude that splitting and merging large 3D images can be done efficiently without relying on complex data formats.
AB - Splitting and merging data is a requirement for many parallel or distributed processing operations. Naive algorithms to split and merge 3D blocks from ultra-high resolution images perform very poorly, as a result of seek times. In contrast, naive algorithms to split and merge 3D slabs perform optimally as seek time is significantly minimized. We introduce and analyze sequential algorithms (Clustered reads, Multiple reads, Clustered writes, and Multiple writes) that leverage memory buffering to address this issue. Clustered reads and Clustered writes, access image chunks only once, but they have to seek in the reconstructed image. Multiple reads and Multiple writes minimize seeks in the reconstructed image, but they access image chunks multiple times. Evaluation on a 3850×3025×3500 brain image shows that our algorithms perform similarly to the optimal configuration provided that enough memory is available. Additionally, Multiple reads supports on-the-fly compression of the merged image transparently but Clustered reads does not, due to its use of negative seeking. We conclude that splitting and merging large 3D images can be done efficiently without relying on complex data formats.
UR - https://www.scopus.com/pages/publications/85047775359
U2 - 10.1109/BigData.2017.8257953
DO - 10.1109/BigData.2017.8257953
M3 - Contribution to conference proceedings
AN - SCOPUS:85047775359
T3 - Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017
SP - 415
EP - 424
BT - Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017
A2 - Nie, Jian-Yun
A2 - Obradovic, Zoran
A2 - Suzumura, Toyotaro
A2 - Ghosh, Rumi
A2 - Nambiar, Raghunath
A2 - Wang, Chonggang
A2 - Zang, Hui
A2 - Baeza-Yates, Ricardo
A2 - Baeza-Yates, Ricardo
A2 - Hu, Xiaohua
A2 - Kepner, Jeremy
A2 - Cuzzocrea, Alfredo
A2 - Tang, Jian
A2 - Toyoda, Masashi
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 5th IEEE International Conference on Big Data, Big Data 2017
Y2 - 11 December 2017 through 14 December 2017
ER -