/orcd/pool/004/jahn/centos7/MITgcm_ifort-mpi-fast_san/verification/tutorial_deep_convection/run ../../../tools/tst_2+2 -mpi All -command "mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv" cmdEXE='mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv' from previous run STDOUT.0000, lastPick=' 3 ckptA' ; iter='3' ; sufx='ckptA' prepare parameter file 'data.tst' : prepare file 'data.tst' : done diff data.tst data 41,43c41 < nIter0=3, < nTimeSteps=4, < # nIter0=0, --- > nIter0=0, 45c43 < # nTimeSteps=3, --- > nTimeSteps=3, 48,49c46,47 < # pChkptFreq=43200., < # chkptFreq=7200., --- > pChkptFreq=43200., > chkptFreq=7200., link back: temp_tst/pickup*.ckptA* rename ckptA -> 0000000003 for all: pickup rnp_loc: pickup.ckptA pickup.0000000003 start-end iter: 3 , 5 , 7 sufix: '0000000003' '0000000005' '0000000007' cmdEXE=mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv ==> START RUN 2 x 2 it [0] MPI startup(): Single-threaded optimized library [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [2] MPI startup(): shm and dapl data transfer modes [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 2 x 2 it listP= pickup rnp_loc: pickup.ckptA pickup.0000000007 move_outp: res_2it ==> START RUN 1iA [0] MPI startup(): Single-threaded optimized library [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [2] MPI startup(): shm and dapl data transfer modes [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 1iA rnp_loc: pickup.ckptA pickup.0000000005 move_outp: res_1iA ==> START RUN 1iB [0] MPI startup(): Single-threaded optimized library [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [2] MPI startup(): shm and dapl data transfer modes [0] MPI startup(): shm and dapl data transfer modes [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 1iB rnp_loc: pickup.ckptA pickup.0000000007 move_outp: res_1iB == compare cg2d_init_res : run 1iA: 7.52109054785187E-01 7.55440213582342E-01 run 1iB: 7.52154208371569E-01 7.56269131161622E-01 run 2it: 7.41853511166679E-01 7.44717865697540E-01 7.47352187115189E-01 7.51585933801657E-01 == compare cg3d_init_res : run 1iA: 8.77398090054373E-01 8.52246251811394E-01 run 1iB: 6.68319155320597E-01 6.70601034606587E-01 run 2it: 6.92731960375701E-01 6.83555436772941E-01 6.75647352383206E-01 6.69247418932785E-01 == diff pickup files : end of 1rst run (2x2 it) & end of 3rd run (2nd 2 it) --> file=pickup.0000000007, listY=.001.001. .001.002. .002.001. diff res_2it/pickup.0000000007.001.001.data res_1iB Binary files res_2it/pickup.0000000007.001.001.data and res_1iB/pickup.0000000007.001.001.data differ Diff outp= 2 (nb-Diff= 5339331 ) ==> stop