num_sent_processes: 4 [0] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile25k/5W100D-0.1000r/shuffled_datasets [1] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile25k/5W100D-0.1000r/shuffled_datasets [2] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile25k/5W100D-0.1000r/shuffled_datasets [3] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile25k/5W100D-0.1000r/shuffled_datasets {'num_shuffle': 100, 'embed_dim': 100, 'context_len': 5, 'nworkers': 6, 'save_freq': -1, 'save_dir': '/scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile25k/5W100D-0.1000r', 'resume': '', 'train_alg': 'cbow', 'min_count': 5, 'neg_samples': 5, 'init_lr': 0.1, 'min_lr': 0.0001, 'lr_mode': 'linear', 'milestones': [], 'hier_softmax': False, 'update_vocab': 'once', 'seed': 0} Using cbow, negative sampling with 5 negative samples [03/12/24-18:21:17] Start training [03/12/24-18:21:17] Building vocabulary [03/12/24-18:21:33] Vocabulary size is 100053 [Shuffling 1] loss 4507652.5000 lr 0.1000 vocab_size 100053 (6.92s/11.79m) [Shuffling 2] loss 4234567.0000 lr 0.0990 vocab_size 100053 (7.05s/11.90m) [Shuffling 3] loss 4127007.7500 lr 0.0980 vocab_size 100053 (7.08s/11.96m) [Shuffling 4] loss 4124251.7500 lr 0.0970 vocab_size 100053 (7.11s/12.00m) [Shuffling 5] loss 3849083.5000 lr 0.0960 vocab_size 100053 (7.07s/12.01m) [Shuffling 6] loss 4106195.2500 lr 0.0950 vocab_size 100053 (6.96s/11.99m) [Shuffling 7] loss 4096337.5000 lr 0.0940 vocab_size 100053 (6.97s/11.97m) [Shuffling 8] loss 4108453.2500 lr 0.0930 vocab_size 100053 (6.97s/11.96m) [Shuffling 9] loss 4187976.5000 lr 0.0920 vocab_size 100053 (6.98s/11.95m) [Shuffling 10] loss 4086286.2500 lr 0.0910 vocab_size 100053 (6.98s/11.95m) [Shuffling 11] loss 4347592.0000 lr 0.0900 vocab_size 100053 (6.96s/11.94m) [Shuffling 12] loss 4206984.5000 lr 0.0890 vocab_size 100053 (7.00s/11.94m) [Shuffling 13] loss 4204487.0000 lr 0.0880 vocab_size 100053 (7.01s/11.94m) [Shuffling 14] loss 4229483.5000 lr 0.0870 vocab_size 100053 (7.00s/11.94m) [Shuffling 15] loss 4269116.5000 lr 0.0860 vocab_size 100053 (7.04s/11.95m) [Shuffling 16] loss 4328766.0000 lr 0.0850 vocab_size 100053 (7.01s/11.95m) [Shuffling 17] loss 4294008.5000 lr 0.0840 vocab_size 100053 (7.00s/11.95m) [Shuffling 18] loss 4182587.0000 lr 0.0830 vocab_size 100053 (7.04s/11.95m) [Shuffling 19] loss 4243706.5000 lr 0.0820 vocab_size 100053 (7.00s/11.95m) [Shuffling 20] loss 4130366.7500 lr 0.0810 vocab_size 100053 (7.09s/11.95m) [Shuffling 21] loss 4264852.0000 lr 0.0800 vocab_size 100053 (7.03s/11.96m) [Shuffling 22] loss 4061219.0000 lr 0.0790 vocab_size 100053 (7.00s/11.95m) [Shuffling 23] loss 4223555.0000 lr 0.0780 vocab_size 100053 (7.05s/11.96m) [Shuffling 24] loss 4186902.7500 lr 0.0770 vocab_size 100053 (6.97s/11.95m) [Shuffling 25] loss 4216053.5000 lr 0.0760 vocab_size 100053 (7.02s/11.95m) [Shuffling 26] loss 4219390.5000 lr 0.0750 vocab_size 100053 (7.04s/11.96m) [Shuffling 27] loss 4068900.5000 lr 0.0740 vocab_size 100053 (7.00s/11.96m) [Shuffling 28] loss 4125015.0000 lr 0.0730 vocab_size 100053 (7.00s/11.96m) [Shuffling 29] loss 4194534.0000 lr 0.0720 vocab_size 100053 (7.05s/11.96m) [Shuffling 30] loss 4207279.0000 lr 0.0710 vocab_size 100053 (7.04s/11.96m) [Shuffling 31] loss 4135075.5000 lr 0.0700 vocab_size 100053 (7.01s/11.96m) [Shuffling 32] loss 4086754.2500 lr 0.0690 vocab_size 100053 (7.04s/11.96m) [Shuffling 33] loss 4255620.5000 lr 0.0680 vocab_size 100053 (7.07s/11.96m) [Shuffling 34] loss 4033249.5000 lr 0.0670 vocab_size 100053 (7.05s/11.96m) [Shuffling 35] loss 4058991.7500 lr 0.0660 vocab_size 100053 (7.05s/11.97m) [Shuffling 36] loss 4138576.0000 lr 0.0650 vocab_size 100053 (7.01s/11.97m) [Shuffling 37] loss 4037081.7500 lr 0.0640 vocab_size 100053 (6.98s/11.96m) [Shuffling 38] loss 4177867.2500 lr 0.0630 vocab_size 100053 (6.97s/11.96m) [Shuffling 39] loss 4041933.0000 lr 0.0620 vocab_size 100053 (7.03s/11.96m) [Shuffling 40] loss 4103785.2500 lr 0.0610 vocab_size 100053 (7.02s/11.96m) [Shuffling 41] loss 4097773.2500 lr 0.0600 vocab_size 100053 (7.01s/11.96m) [Shuffling 42] loss 4090165.5000 lr 0.0590 vocab_size 100053 (6.98s/11.96m) [Shuffling 43] loss 4019851.7500 lr 0.0580 vocab_size 100053 (7.03s/11.96m) [Shuffling 44] loss 4114673.5000 lr 0.0570 vocab_size 100053 (7.02s/11.96m) [Shuffling 45] loss 4113064.7500 lr 0.0560 vocab_size 100053 (7.01s/11.96m) [Shuffling 46] loss 4022677.2500 lr 0.0550 vocab_size 100053 (7.01s/11.96m) [Shuffling 47] loss 4126272.2500 lr 0.0540 vocab_size 100053 (6.90s/11.96m) [Shuffling 48] loss 4230129.0000 lr 0.0530 vocab_size 100053 (6.96s/11.96m) [Shuffling 49] loss 4213341.0000 lr 0.0520 vocab_size 100053 (6.92s/11.95m) [Shuffling 50] loss 4272512.5000 lr 0.0510 vocab_size 100053 (6.93s/11.95m) [Shuffling 51] loss 4259229.5000 lr 0.0500 vocab_size 100053 (6.93s/11.95m) [Shuffling 52] loss 4066334.2500 lr 0.0491 vocab_size 100053 (7.01s/11.95m) [Shuffling 53] loss 4008722.2500 lr 0.0481 vocab_size 100053 (6.99s/11.95m) [Shuffling 54] loss 4018437.5000 lr 0.0471 vocab_size 100053 (6.95s/11.94m) [Shuffling 55] loss 4109068.2500 lr 0.0461 vocab_size 100053 (7.01s/11.94m) [Shuffling 56] loss 4010704.0000 lr 0.0451 vocab_size 100053 (6.98s/11.94m) [Shuffling 57] loss 4049174.0000 lr 0.0441 vocab_size 100053 (6.93s/11.94m) [Shuffling 58] loss 3974129.2500 lr 0.0431 vocab_size 100053 (7.00s/11.94m) [Shuffling 59] loss 4041320.2500 lr 0.0421 vocab_size 100053 (6.99s/11.94m) [Shuffling 60] loss 4061195.2500 lr 0.0411 vocab_size 100053 (6.98s/11.94m) [Shuffling 61] loss 3993507.5000 lr 0.0401 vocab_size 100053 (6.99s/11.94m) [Shuffling 62] loss 3985347.7500 lr 0.0391 vocab_size 100053 (7.00s/11.94m) [Shuffling 63] loss 3939285.0000 lr 0.0381 vocab_size 100053 (7.01s/11.94m) [Shuffling 64] loss 3961290.0000 lr 0.0371 vocab_size 100053 (7.05s/11.94m) [Shuffling 65] loss 4029970.0000 lr 0.0361 vocab_size 100053 (7.04s/11.94m) [Shuffling 66] loss 3783202.2500 lr 0.0351 vocab_size 100053 (7.20s/11.95m) [Shuffling 67] loss 3832117.0000 lr 0.0341 vocab_size 100053 (6.99s/11.95m) [Shuffling 68] loss 3893180.7500 lr 0.0331 vocab_size 100053 (7.03s/11.95m) [Shuffling 69] loss 3969909.5000 lr 0.0321 vocab_size 100053 (7.00s/11.95m) [Shuffling 70] loss 4050477.7500 lr 0.0311 vocab_size 100053 (7.03s/11.95m) [Shuffling 71] loss 4004411.7500 lr 0.0301 vocab_size 100053 (7.08s/11.95m) [Shuffling 72] loss 3999495.5000 lr 0.0291 vocab_size 100053 (6.98s/11.95m) [Shuffling 73] loss 3918191.7500 lr 0.0281 vocab_size 100053 (7.06s/11.95m) [Shuffling 74] loss 3940800.7500 lr 0.0271 vocab_size 100053 (6.99s/11.95m) [Shuffling 75] loss 4039763.7500 lr 0.0261 vocab_size 100053 (7.02s/11.95m) [Shuffling 76] loss 3899588.7500 lr 0.0251 vocab_size 100053 (7.04s/11.95m) [Shuffling 77] loss 4015149.7500 lr 0.0241 vocab_size 100053 (7.08s/11.95m) [Shuffling 78] loss 3982488.7500 lr 0.0231 vocab_size 100053 (7.01s/11.95m) [Shuffling 79] loss 3858087.7500 lr 0.0221 vocab_size 100053 (7.10s/11.95m) [Shuffling 80] loss 3962151.7500 lr 0.0211 vocab_size 100053 (7.09s/11.96m) [Shuffling 81] loss 3896454.0000 lr 0.0201 vocab_size 100053 (7.05s/11.96m) [Shuffling 82] loss 3909754.0000 lr 0.0191 vocab_size 100053 (7.06s/11.96m) [Shuffling 83] loss 3987374.0000 lr 0.0181 vocab_size 100053 (7.02s/11.96m) [Shuffling 84] loss 3887456.0000 lr 0.0171 vocab_size 100053 (7.05s/11.96m) [Shuffling 85] loss 3901742.5000 lr 0.0161 vocab_size 100053 (7.09s/11.96m) [Shuffling 86] loss 3932510.2500 lr 0.0151 vocab_size 100053 (7.06s/11.96m) [Shuffling 87] loss 3845282.0000 lr 0.0141 vocab_size 100053 (7.09s/11.96m) [Shuffling 88] loss 3823055.2500 lr 0.0131 vocab_size 100053 (7.03s/11.96m) [Shuffling 89] loss 3887062.5000 lr 0.0121 vocab_size 100053 (7.06s/11.96m) [Shuffling 90] loss 3860628.0000 lr 0.0111 vocab_size 100053 (7.01s/11.96m) [Shuffling 91] loss 3756546.2500 lr 0.0101 vocab_size 100053 (7.13s/11.97m) [Shuffling 92] loss 3856576.7500 lr 0.0091 vocab_size 100053 (7.06s/11.97m) [Shuffling 93] loss 3860130.2500 lr 0.0081 vocab_size 100053 (7.05s/11.97m) [Shuffling 94] loss 3934938.7500 lr 0.0071 vocab_size 100053 (7.05s/11.97m) [Shuffling 95] loss 3902248.2500 lr 0.0061 vocab_size 100053 (7.11s/11.97m) [Shuffling 96] loss 3861449.7500 lr 0.0051 vocab_size 100053 (7.03s/11.97m) [Shuffling 97] loss 3862289.2500 lr 0.0041 vocab_size 100053 (6.99s/11.97m) [Shuffling 98] loss 3837722.2500 lr 0.0031 vocab_size 100053 (7.08s/11.97m) [Shuffling 99] loss 3892505.0000 lr 0.0021 vocab_size 100053 (7.02s/11.97m) [Shuffling 100] loss 3856267.2500 lr 0.0011 vocab_size 100053 (6.95s/11.97m) [03/12/24-18:33:16] Training finished, training Time 11.97m