/orcd/pool/004/jahn/centos7/MITgcm_ifort-mpi-fast_san/verification/exp4/tr_run.stevens ../../../tools/tst_2+2 -mpi All -command "mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv" cmdEXE='mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv' from previous run STDOUT.0000, lastPick=' 10 ckptA' ; iter='10' ; sufx='ckptA' prepare parameter file 'data.tst' : prepare file 'data.tst' : done diff data.tst data 48,51c48,49 < nIter0=10, < nTimeSteps=4, < # nIter0=0, < # nTimeSteps=10, --- > nIter0=0, > nTimeSteps=10, 56,57c54,55 < # pChkptFreq=0.0, < # chkptFreq=0.0, --- > pChkptFreq=0.0, > chkptFreq=0.0, link back: temp_tst/pickup*.ckptA* rename ckptA -> 0000000010 for all: pickup pickup_stevensE pickup_stevensW rnp_loc: pickup.ckptA pickup.0000000010 rnp_loc: pickup_stevensE.ckptA pickup_stevensE.0000000010 rnp_loc: pickup_stevensW.ckptA pickup_stevensW.0000000010 start-end iter: 10 , 12 , 14 sufix: '0000000010' '0000000012' '0000000014' cmdEXE=mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv ==> START RUN 2 x 2 it [0] MPI startup(): Single-threaded optimized library [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [2] MPI startup(): shm and dapl data transfer modes [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 2 x 2 it listP= pickup pickup_stevensE pickup_stevensW rnp_loc: pickup.ckptA pickup.0000000014 rnp_loc: pickup_stevensE.ckptA pickup_stevensE.0000000014 rnp_loc: pickup_stevensW.ckptA pickup_stevensW.0000000014 move_outp: res_2it ==> START RUN 1iA [0] MPI startup(): Single-threaded optimized library [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [2] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 1iA rnp_loc: pickup.ckptA pickup.0000000012 rnp_loc: pickup_stevensE.ckptA pickup_stevensE.0000000012 rnp_loc: pickup_stevensW.ckptA pickup_stevensW.0000000012 move_outp: res_1iA ==> START RUN 1iB [0] MPI startup(): Single-threaded optimized library [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [2] MPI startup(): shm and dapl data transfer modes [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 1iB rnp_loc: pickup.ckptA pickup.0000000014 rnp_loc: pickup_stevensE.ckptA pickup_stevensE.0000000014 rnp_loc: pickup_stevensW.ckptA pickup_stevensW.0000000014 move_outp: res_1iB == compare cg2d_init_res : run 1iA: 9.76365402997813E-01 1.13880865250143E+00 run 1iB: 7.87649416668300E-01 7.63262867009007E-01 run 2it: 9.76365402997813E-01 1.13892048979103E+00 7.93469285044314E-01 7.71269799991673E-01 == diff pickup files : end of 1rst run (2x2 it) & end of 3rd run (2nd 2 it) --> file=pickup.0000000014, listY=.001.001. .001.002. .002.001. .002.002. diff res_2it/pickup.0000000014.001.001.data res_1iB Binary files res_2it/pickup.0000000014.001.001.data and res_1iB/pickup.0000000014.001.001.data differ Diff outp= 2 (nb-Diff= 193809 ) ==> stop