num_sent_processes: 4 [2] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile1k/50W100D-0.5000r/shuffled_datasets {'num_shuffle': 100, 'embed_dim': 100, 'context_len': 50, 'nworkers': 6, 'save_freq': -1, 'save_dir': '/scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile1k/50W100D-0.5000r', 'resume': '', 'train_alg': 'cbow', 'min_count': 5, 'neg_samples': 5, 'init_lr': 0.5, 'min_lr': 0.0001, 'lr_mode': 'linear', 'milestones': [], 'hier_softmax': False, 'update_vocab': 'once', 'seed': 0} [3] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile1k/50W100D-0.5000r/shuffled_datasets [1] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile1k/50W100D-0.5000r/shuffled_datasets [0] Creating shuffled datasets in /scratch/gz5hp/tfbs_region2vec_models/expr_universe_tile1k/50W100D-0.5000r/shuffled_datasets Using cbow, negative sampling with 5 negative samples [03/12/24-21:47:53] Start training [03/12/24-21:47:53] Building vocabulary [03/12/24-21:48:27] Vocabulary size is 426964 [Shuffling 1] loss 1681190.7500 lr 0.5000 vocab_size 426964 (16.66s/28.34m) [Shuffling 2] loss 79333.1875 lr 0.4950 vocab_size 426964 (17.10s/28.72m) [Shuffling 3] loss 45013.6055 lr 0.4900 vocab_size 426964 (17.16s/28.87m) [Shuffling 4] loss 34603.2344 lr 0.4850 vocab_size 426964 (17.12s/28.93m) [Shuffling 5] loss 26196.6484 lr 0.4800 vocab_size 426964 (17.02s/28.94m) [Shuffling 6] loss 20363.8867 lr 0.4750 vocab_size 426964 (17.19s/28.99m) [Shuffling 7] loss 18539.0312 lr 0.4700 vocab_size 426964 (16.78s/28.93m) [Shuffling 8] loss 16394.7168 lr 0.4650 vocab_size 426964 (16.63s/28.85m) [Shuffling 9] loss 14452.3154 lr 0.4600 vocab_size 426964 (16.96s/28.85m) [Shuffling 10] loss 12505.0752 lr 0.4550 vocab_size 426964 (16.74s/28.81m) [Shuffling 11] loss 11691.3926 lr 0.4500 vocab_size 426964 (17.28s/28.86m) [Shuffling 12] loss 11604.9561 lr 0.4450 vocab_size 426964 (17.45s/28.93m) [Shuffling 13] loss 10940.3662 lr 0.4400 vocab_size 426964 (17.30s/28.97m) [Shuffling 14] loss 10331.3906 lr 0.4350 vocab_size 426964 (17.00s/28.96m) [Shuffling 15] loss 8524.3896 lr 0.4300 vocab_size 426964 (17.65s/29.03m) [Shuffling 16] loss 8577.8291 lr 0.4250 vocab_size 426964 (17.26s/29.05m) [Shuffling 17] loss 8713.9990 lr 0.4200 vocab_size 426964 (17.10s/29.06m) [Shuffling 18] loss 8713.4541 lr 0.4150 vocab_size 426964 (17.43s/29.09m) [Shuffling 19] loss 7013.8315 lr 0.4100 vocab_size 426964 (17.30s/29.10m) [Shuffling 20] loss 6380.1919 lr 0.4050 vocab_size 426964 (17.23s/29.11m) [Shuffling 21] loss 6416.1997 lr 0.4000 vocab_size 426964 (16.82s/29.09m) [Shuffling 22] loss 7080.8408 lr 0.3950 vocab_size 426964 (17.27s/29.10m) [Shuffling 23] loss 6494.0420 lr 0.3900 vocab_size 426964 (17.01s/29.09m) [Shuffling 24] loss 6214.9067 lr 0.3850 vocab_size 426964 (17.29s/29.11m) [Shuffling 25] loss 6372.5132 lr 0.3800 vocab_size 426964 (16.64s/29.07m) [Shuffling 26] loss 6648.1924 lr 0.3750 vocab_size 426964 (17.11s/29.07m) [Shuffling 27] loss 5913.6265 lr 0.3700 vocab_size 426964 (17.89s/29.12m) [Shuffling 28] loss 6692.0522 lr 0.3650 vocab_size 426964 (16.62s/29.09m) [Shuffling 29] loss 6133.0879 lr 0.3600 vocab_size 426964 (17.21s/29.10m) [Shuffling 30] loss 5183.6641 lr 0.3550 vocab_size 426964 (17.16s/29.10m) [Shuffling 31] loss 5974.9526 lr 0.3500 vocab_size 426964 (16.96s/29.09m) [Shuffling 32] loss 6434.0898 lr 0.3450 vocab_size 426964 (17.06s/29.09m) [Shuffling 33] loss 5920.0601 lr 0.3400 vocab_size 426964 (16.84s/29.08m) [Shuffling 34] loss 5818.8306 lr 0.3350 vocab_size 426964 (17.10s/29.08m) [Shuffling 35] loss 5724.7617 lr 0.3300 vocab_size 426964 (17.71s/29.11m) [Shuffling 36] loss 5238.1914 lr 0.3250 vocab_size 426964 (16.86s/29.10m) [Shuffling 37] loss 4776.3203 lr 0.3200 vocab_size 426964 (17.77s/29.12m) [Shuffling 38] loss 4934.2236 lr 0.3150 vocab_size 426964 (17.93s/29.16m) [Shuffling 39] loss 5455.7231 lr 0.3100 vocab_size 426964 (17.23s/29.16m) [Shuffling 40] loss 5682.9644 lr 0.3050 vocab_size 426964 (16.77s/29.15m) [Shuffling 41] loss 5013.7373 lr 0.3000 vocab_size 426964 (17.79s/29.17m) [Shuffling 42] loss 4462.6167 lr 0.2950 vocab_size 426964 (17.47s/29.19m) [Shuffling 43] loss 5025.7544 lr 0.2900 vocab_size 426964 (18.38s/29.23m) [Shuffling 44] loss 5120.1348 lr 0.2850 vocab_size 426964 (16.96s/29.22m) [Shuffling 45] loss 4018.3147 lr 0.2800 vocab_size 426964 (17.69s/29.24m) [Shuffling 46] loss 4755.0518 lr 0.2750 vocab_size 426964 (17.24s/29.24m) [Shuffling 47] loss 4548.2158 lr 0.2700 vocab_size 426964 (17.80s/29.27m) [Shuffling 48] loss 4410.8218 lr 0.2650 vocab_size 426964 (17.23s/29.27m) [Shuffling 49] loss 4203.8110 lr 0.2600 vocab_size 426964 (16.69s/29.25m) [Shuffling 50] loss 4164.9233 lr 0.2550 vocab_size 426964 (16.78s/29.23m) [Shuffling 51] loss 4376.9951 lr 0.2500 vocab_size 426964 (17.28s/29.24m) [Shuffling 52] loss 4231.3833 lr 0.2451 vocab_size 426964 (16.30s/29.21m) [Shuffling 53] loss 4018.1565 lr 0.2401 vocab_size 426964 (16.81s/29.20m) [Shuffling 54] loss 3610.4912 lr 0.2351 vocab_size 426964 (18.96s/29.25m) [Shuffling 55] loss 3911.8655 lr 0.2301 vocab_size 426964 (18.52s/29.29m) [Shuffling 56] loss 4281.8389 lr 0.2251 vocab_size 426964 (16.41s/29.27m) [Shuffling 57] loss 3558.2454 lr 0.2201 vocab_size 426964 (17.44s/29.27m) [Shuffling 58] loss 4008.1250 lr 0.2151 vocab_size 426964 (17.15s/29.27m) [Shuffling 59] loss 3695.6187 lr 0.2101 vocab_size 426964 (17.37s/29.28m) [Shuffling 60] loss 4216.2407 lr 0.2051 vocab_size 426964 (16.50s/29.26m) [Shuffling 61] loss 3721.5969 lr 0.2001 vocab_size 426964 (17.10s/29.25m) [Shuffling 62] loss 3401.8840 lr 0.1951 vocab_size 426964 (17.45s/29.26m) [Shuffling 63] loss 3688.9075 lr 0.1901 vocab_size 426964 (16.65s/29.24m) [Shuffling 64] loss 3780.9653 lr 0.1851 vocab_size 426964 (17.00s/29.24m) [Shuffling 65] loss 3700.3113 lr 0.1801 vocab_size 426964 (17.38s/29.24m) [Shuffling 66] loss 3304.5930 lr 0.1751 vocab_size 426964 (17.73s/29.26m) [Shuffling 67] loss 3679.8252 lr 0.1701 vocab_size 426964 (16.20s/29.23m) [Shuffling 68] loss 3655.6108 lr 0.1651 vocab_size 426964 (17.50s/29.24m) [Shuffling 69] loss 3506.2988 lr 0.1601 vocab_size 426964 (16.73s/29.23m) [Shuffling 70] loss 3801.7036 lr 0.1551 vocab_size 426964 (16.55s/29.21m) [Shuffling 71] loss 3399.5833 lr 0.1501 vocab_size 426964 (16.60s/29.20m) [Shuffling 72] loss 3606.1384 lr 0.1451 vocab_size 426964 (17.12s/29.20m) [Shuffling 73] loss 3206.7070 lr 0.1401 vocab_size 426964 (17.92s/29.21m) [Shuffling 74] loss 3123.6514 lr 0.1351 vocab_size 426964 (17.32s/29.22m) [Shuffling 75] loss 3305.0503 lr 0.1301 vocab_size 426964 (17.18s/29.22m) [Shuffling 76] loss 3286.6538 lr 0.1251 vocab_size 426964 (17.18s/29.22m) [Shuffling 77] loss 3399.1069 lr 0.1201 vocab_size 426964 (16.98s/29.21m) [Shuffling 78] loss 3159.1050 lr 0.1151 vocab_size 426964 (17.64s/29.22m) [Shuffling 79] loss 2966.9026 lr 0.1101 vocab_size 426964 (17.31s/29.22m) [Shuffling 80] loss 3513.5813 lr 0.1051 vocab_size 426964 (16.92s/29.22m) [Shuffling 81] loss 3398.9160 lr 0.1001 vocab_size 426964 (16.65s/29.21m) [Shuffling 82] loss 3032.8672 lr 0.0951 vocab_size 426964 (16.83s/29.20m) [Shuffling 83] loss 3866.9141 lr 0.0901 vocab_size 426964 (16.84s/29.19m) [Shuffling 84] loss 3332.4636 lr 0.0851 vocab_size 426964 (16.66s/29.18m) [Shuffling 85] loss 2766.4492 lr 0.0801 vocab_size 426964 (17.71s/29.19m) [Shuffling 86] loss 3263.3838 lr 0.0751 vocab_size 426964 (16.75s/29.19m) [Shuffling 87] loss 3224.5500 lr 0.0701 vocab_size 426964 (17.28s/29.19m) [Shuffling 88] loss 3438.5249 lr 0.0651 vocab_size 426964 (17.38s/29.19m) [Shuffling 89] loss 3224.4751 lr 0.0601 vocab_size 426964 (18.01s/29.21m) [Shuffling 90] loss 3156.4155 lr 0.0551 vocab_size 426964 (17.69s/29.22m) [Shuffling 91] loss 2960.3887 lr 0.0501 vocab_size 426964 (16.38s/29.20m) [Shuffling 92] loss 2864.6504 lr 0.0451 vocab_size 426964 (16.76s/29.19m) [Shuffling 93] loss 3200.2615 lr 0.0401 vocab_size 426964 (17.98s/29.21m) [Shuffling 94] loss 3416.3862 lr 0.0351 vocab_size 426964 (17.04s/29.21m) [Shuffling 95] loss 2986.9299 lr 0.0301 vocab_size 426964 (16.87s/29.20m) [Shuffling 96] loss 3229.6216 lr 0.0251 vocab_size 426964 (16.55s/29.19m) [Shuffling 97] loss 3194.5356 lr 0.0201 vocab_size 426964 (17.53s/29.20m) [Shuffling 98] loss 3157.0813 lr 0.0151 vocab_size 426964 (16.21s/29.18m) [Shuffling 99] loss 3677.9600 lr 0.0101 vocab_size 426964 (16.18s/29.16m) [Shuffling 100] loss 3135.7378 lr 0.0051 vocab_size 426964 (33.24s/29.43m) [03/12/24-22:17:19] Training finished, training Time 29.43m