One flaw in the idea: the daemons are not MPI processes, and therefore have no way to run an MPI I/O operation.
On Jun 13, 2012, at 5:40 AM, Xuan Wang wrote:
> I have an idea about using database to support a kind of semi-automatic optimized parallel I/O operations and want to know if it is realizable or not. Hope you guys can give me more advices and point out the shortage of the idea. Thank you all.
> As the performance of the parallel I/O depends on the parallel I/O algorithm, the file storage in file system, the number of processes used for I/O and so on, we can use the MPI hints to control the parameters manually. But sometime, the client or the people who call the I/O operation dont know which parameters are the best.
> Therefore, we think about using the data warehouse and an I/O monitor to realize the optimization phase. Please take a look at the attached picture first.
> Process explanations:
> 1. The client sends the I/O commends with hints (optional) to deamon. The select model will decide if it is necessary to call the I/O database in order to get the optimized I/O operation strategy.
> 2. If yes, the select model sends the I/O commends with those parameters, which can be used to choose the optimized I/O algorithm, to the knowledge base or database.
> 3 & 4. The select model gets the optimized algorithm and runs the I/O operation.
> 5 & 6. During the I/O operation, the monitor will gather the performance related information and sends it to the data warehouse, which is used to analyze the performance of the optimized algorithm and support the semi-automatic optimization.
> These are the basic thought about the whole process. Please be free to ask any details about this system/concept. I will try my best to explain it.
> I am happy if someone can take part in the discussion.
> Best Regards!
> Xuan Wang<1.PNG>_______________________________________________
> users mailing list