/orcd/pool/004/jahn/centos7/MITgcm_ifort-mpi-fast_san/verification/aim.5l_LatLon/run ../../../tools/tst_2+2 -mpi All -command "mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv" cmdEXE='mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv' from previous run STDOUT.0000, lastPick=' 69130 ckptA' ; iter='69130' ; sufx='ckptA' prepare parameter file 'data.tst' : prepare file 'data.tst' : done diff data.tst data 39,42c39,40 < nIter0=69130, < nTimeSteps=4, < # niter0=69120, < # nTimeSteps=10, --- > niter0=69120, > nTimeSteps=10, 48,50c46,48 < # pickupStrictlyMatch=.FALSE., < # pChkptFreq=2592000.0, < # chkptFreq=0.0, --- > pickupStrictlyMatch=.FALSE., > pChkptFreq=2592000.0, > chkptFreq=0.0, link back: temp_tst/pickup*.ckptA* rename ckptA -> 0000069130 for all: pickup rnp_loc: pickup.ckptA pickup.0000069130 start-end iter: 69130 , 69132 , 69134 sufix: '0000069130' '0000069132' '0000069134' cmdEXE=mpirun -env I_MPI_DEBUG 2 -n 4 ./mitgcmuv ==> START RUN 2 x 2 it [0] MPI startup(): Single-threaded optimized library [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [2] MPI startup(): shm and dapl data transfer modes [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 2 x 2 it listP= pickup rnp_loc: pickup.ckptA pickup.0000069134 move_outp: res_2it ==> START RUN 1iA [0] MPI startup(): Single-threaded optimized library [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [2] MPI startup(): shm and dapl data transfer modes [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 1iA rnp_loc: pickup.ckptA pickup.0000069132 move_outp: res_1iA ==> START RUN 1iB [0] MPI startup(): Single-threaded optimized library [2] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [0] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [1] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [2] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [2] MPI startup(): shm and dapl data transfer modes [0] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [0] MPI startup(): shm and dapl data transfer modes [1] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [1] MPI startup(): shm and dapl data transfer modes [3] DAPL startup(): trying to open DAPL provider from I_MPI_DAPL_PROVIDER: ofa-v2-mlx5_0-1u [3] MPI startup(): DAPL provider ofa-v2-mlx5_0-1u [3] MPI startup(): shm and dapl data transfer modes NORMAL END NORMAL END NORMAL END NORMAL END ==> END RUN 1iB rnp_loc: pickup.ckptA pickup.0000069134 move_outp: res_1iB == compare cg2d_init_res : run 1iA: 4.29463302372147E+03 4.30140713561873E+03 run 1iB: 4.30858192875855E+03 4.31602771893636E+03 run 2it: 4.28152477963594E+03 4.28689365058454E+03 4.29258963194467E+03 4.29860322088137E+03 == diff pickup files : end of 1rst run (2x2 it) & end of 3rd run (2nd 2 it) --> file=pickup.0000069134, listY=.001.001. .001.002. .001.003. diff res_2it/pickup.0000069134.001.001.data res_1iB Binary files res_2it/pickup.0000069134.001.001.data and res_1iB/pickup.0000069134.001.001.data differ Diff outp= 2 (nb-Diff= 78046 ) ==> stop