{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":691604054,"defaultBranch":"master","name":"reinforcement_learning","ownerLogin":"EdwardLeeMacau","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-09-14T14:10:12.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/34258587?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1699175367.0","currentOid":""},"activityList":{"items":[{"before":"cf44d6e4f3c56d0be8a226aeb161a4d918bf7a94","after":"3fb2139bbd1c6ff02733025a37dc95cd167bef93","ref":"refs/heads/master","pushedAt":"2023-11-21T06:02:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Impl RL for 2048\n\nSquashed commit of the following:\n\ncommit e3f2bf9bb62bb1870d6a6ea71e14bedc3122e51c\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Thu Nov 9 00:52:21 2023 +0800\n\n    Optimize speed\n\ncommit c25afdd7e8be5f0f923006977f81ea8a8a0ea75f\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Thu Nov 9 00:52:16 2023 +0800\n\n    Add Makefile\n\ncommit abdd30a7e6664c41ee8e0c875a25aa318fa6a266\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Nov 8 14:22:19 2023 +0800\n\n    Optimize speed\n\ncommit eda81b62272c160058f3b375b7d468a99b8f5ba1\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Nov 8 14:14:06 2023 +0800\n\n    Optimize speed\n\ncommit eb71738c114b2f9a11991d94ef00f61d50226285\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Nov 8 13:56:17 2023 +0800\n\n    Revise gridworld\n\ncommit 1a377d6afcb4f0228069485d5375e074311fd7a1\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Nov 8 11:44:03 2023 +0800\n\n    Add PPOv29\n\ncommit 0255a73287cd367ebaea099ebacfa85800a708c1\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Nov 8 11:42:17 2023 +0800\n\n    Add PPOv28\n\ncommit a73825129a08d8ba598d63782d47fc8dbd4598eb\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Nov 8 11:39:08 2023 +0800\n\n    Revise eval.py\n\ncommit 4c124853e016a6a5a9bb3e2cc19f7b18f77296a2\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Nov 8 00:14:28 2023 +0800\n\n    Add PPOv27 conv-grey-code\n\ncommit a977a04527121edbaa3413f59b19dcc2c87e457a\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Sun Nov 5 11:39:53 2023 +0800\n\n    Reward Shaping and PPOv26\n\n    Add PPOv26\n\ncommit e3e0207a2e884decbc90037c28b9b34cc1fc4117\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Thu Nov 2 22:02:07 2023 +0800\n\n    Draft GridWorldv2","shortMessageHtmlLink":"Impl RL for 2048"}},{"before":"e3f2bf9bb62bb1870d6a6ea71e14bedc3122e51c","after":"4f877a43a472495e157d776876463d2720a8e069","ref":"refs/heads/hw03","pushedAt":"2023-11-08T17:01:24.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Update eval.py","shortMessageHtmlLink":"Update eval.py"}},{"before":"abdd30a7e6664c41ee8e0c875a25aa318fa6a266","after":"e3f2bf9bb62bb1870d6a6ea71e14bedc3122e51c","ref":"refs/heads/hw03","pushedAt":"2023-11-08T16:58:00.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Optimize speed","shortMessageHtmlLink":"Optimize speed"}},{"before":"1a377d6afcb4f0228069485d5375e074311fd7a1","after":"abdd30a7e6664c41ee8e0c875a25aa318fa6a266","ref":"refs/heads/hw03","pushedAt":"2023-11-08T06:22:26.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Optimize speed","shortMessageHtmlLink":"Optimize speed"}},{"before":"de94731b73f80c7a944f7a487b03bb977b4d63b5","after":"1a377d6afcb4f0228069485d5375e074311fd7a1","ref":"refs/heads/hw03","pushedAt":"2023-11-08T05:55:59.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv29","shortMessageHtmlLink":"Add PPOv29"}},{"before":"29f6d6f51cdea4b276bd97b7cf79d93b3a32dd73","after":"de94731b73f80c7a944f7a487b03bb977b4d63b5","ref":"refs/heads/hw03","pushedAt":"2023-11-08T05:27:48.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv29","shortMessageHtmlLink":"Add PPOv29"}},{"before":"7fe2b131a1ee3bfdb54c1f7a459f76a6c9fd2c31","after":"29f6d6f51cdea4b276bd97b7cf79d93b3a32dd73","ref":"refs/heads/hw03","pushedAt":"2023-11-07T15:27:03.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Update eval script","shortMessageHtmlLink":"Update eval script"}},{"before":"134955cf4568865908152427042bb8aa99153009","after":"7fe2b131a1ee3bfdb54c1f7a459f76a6c9fd2c31","ref":"refs/heads/hw03","pushedAt":"2023-11-07T14:21:59.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Init lander.py","shortMessageHtmlLink":"Init lander.py"}},{"before":"dfbc58aa9d99b6eeb6d3713109ce9883fcce4dc4","after":"134955cf4568865908152427042bb8aa99153009","ref":"refs/heads/hw03","pushedAt":"2023-11-07T13:24:07.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv26","shortMessageHtmlLink":"Add PPOv26"}},{"before":"93ae035801bf121add5a6927077229bdef0f6b7b","after":"dfbc58aa9d99b6eeb6d3713109ce9883fcce4dc4","ref":"refs/heads/hw03","pushedAt":"2023-11-07T12:44:42.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv24 train.py fine-tune arch","shortMessageHtmlLink":"Add PPOv24 train.py fine-tune arch"}},{"before":"387176042c162a20b645ee58c9d6d3d68e74bf91","after":"93ae035801bf121add5a6927077229bdef0f6b7b","ref":"refs/heads/hw03","pushedAt":"2023-11-07T12:41:37.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv22 train.py re-impl architecture","shortMessageHtmlLink":"Add PPOv22 train.py re-impl architecture"}},{"before":"0b334a586e978692a24bbdd3d64d83a06c59da1c","after":"387176042c162a20b645ee58c9d6d3d68e74bf91","ref":"refs/heads/hw03","pushedAt":"2023-11-07T12:36:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv22 train.py re-impl architecture","shortMessageHtmlLink":"Add PPOv22 train.py re-impl architecture"}},{"before":"2cc2585ac900d883120febf29ad1265a8b2af9f9","after":"0b334a586e978692a24bbdd3d64d83a06c59da1c","ref":"refs/heads/hw03","pushedAt":"2023-11-07T09:22:20.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv16~18 reward weight","shortMessageHtmlLink":"Add PPOv16~18 reward weight"}},{"before":"a6edcd5902ae1849d6f4752db6e2d4ee046216d8","after":"2cc2585ac900d883120febf29ad1265a8b2af9f9","ref":"refs/heads/hw03","pushedAt":"2023-11-07T03:51:11.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv11 train.py","shortMessageHtmlLink":"Add PPOv11 train.py"}},{"before":"3b4a452bea6663039bb02111ba28857c7386ccc0","after":"a6edcd5902ae1849d6f4752db6e2d4ee046216d8","ref":"refs/heads/hw03","pushedAt":"2023-11-07T03:10:26.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv9 train.py","shortMessageHtmlLink":"Add PPOv9 train.py"}},{"before":"ad80b650118468360bfec090f7a969830e3adc87","after":"3b4a452bea6663039bb02111ba28857c7386ccc0","ref":"refs/heads/hw03","pushedAt":"2023-11-06T16:13:42.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Add PPOv3 train.py","shortMessageHtmlLink":"Add PPOv3 train.py"}},{"before":"2d35f35cb6c55759b0a3143e491000bda2915e1e","after":"ad80b650118468360bfec090f7a969830e3adc87","ref":"refs/heads/hw03","pushedAt":"2023-11-06T15:09:39.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Adjust reward strategy","shortMessageHtmlLink":"Adjust reward strategy"}},{"before":"1a2dba02f5b55a3927b8712a22a58ff9fcbd5f85","after":"2d35f35cb6c55759b0a3143e491000bda2915e1e","ref":"refs/heads/hw03","pushedAt":"2023-11-06T09:11:35.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Enlarge model depth","shortMessageHtmlLink":"Enlarge model depth"}},{"before":"a80a9bec3a4c44c949b3a215d53cc7433b2b937c","after":"1a2dba02f5b55a3927b8712a22a58ff9fcbd5f85","ref":"refs/heads/hw03","pushedAt":"2023-11-05T16:56:23.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Rollback to init code","shortMessageHtmlLink":"Rollback to init code"}},{"before":null,"after":"a80a9bec3a4c44c949b3a215d53cc7433b2b937c","ref":"refs/heads/hw03","pushedAt":"2023-11-05T09:09:27.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Draft FeatureExtractor","shortMessageHtmlLink":"Draft FeatureExtractor"}},{"before":"2b853b98de86937bfc524b04e296957e5f54a781","after":null,"ref":"refs/heads/hw02","pushedAt":"2023-11-02T12:10:37.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"}},{"before":"1819197e40e9df90e080c352f29c1fc97006e0c4","after":"cf44d6e4f3c56d0be8a226aeb161a4d918bf7a94","ref":"refs/heads/master","pushedAt":"2023-11-02T12:10:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Initialize game (RL Fall 2023)","shortMessageHtmlLink":"Initialize game (RL Fall 2023)"}},{"before":"6da5b8eee81322fdaa86452209c13d13e53b9fda","after":"1819197e40e9df90e080c352f29c1fc97006e0c4","ref":"refs/heads/master","pushedAt":"2023-10-19T13:12:11.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Impl Model-Free Control\n\nSquashed commit of the following:\n\ncommit 66ea3b34155546d219c530bfdfcd67b1d2977860\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Thu Oct 19 21:09:09 2023 +0800\n\n    Update report\n\ncommit 1f9f5231d0a9fac62cbcfc7aaa66d739474fd642\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Wed Oct 18 23:16:39 2023 +0800\n\n    Add code to plot graph\n\ncommit 2b853b98de86937bfc524b04e296957e5f54a781\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Mon Oct 16 21:18:24 2023 +0800\n\n    Fix bug in Q-Learning and improve perf\n\ncommit 4c506473408a436b46b8ff7433da0a3128639fa1\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Sat Oct 14 13:08:37 2023 +0800\n\n    Finish Q-Learning\n\ncommit d8e3862a99199740714c5435bff873a132c40926\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Sat Oct 14 12:39:49 2023 +0800\n\n    Finish TD-0 Prediction with Greedy Improvement\n\ncommit 59825789b46d6411a9982785bce93cae55077db7\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Sat Oct 14 12:18:40 2023 +0800\n\n    Finish MC Prediction with Greedy Improvement\n\ncommit 86fd76d7da7a6c4940c8c7f7c3c422648d507d36\nAuthor: EdwardLeeMacau <b05901119@ntu.edu.tw>\nDate:   Sat Oct 14 11:57:10 2023 +0800\n\n    Removing trailing spaces","shortMessageHtmlLink":"Impl Model-Free Control"}},{"before":"4c506473408a436b46b8ff7433da0a3128639fa1","after":"2b853b98de86937bfc524b04e296957e5f54a781","ref":"refs/heads/hw02","pushedAt":"2023-10-16T13:18:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Fix bug in Q-Learning and improve perf","shortMessageHtmlLink":"Fix bug in Q-Learning and improve perf"}},{"before":null,"after":"4c506473408a436b46b8ff7433da0a3128639fa1","ref":"refs/heads/hw02","pushedAt":"2023-10-16T12:26:41.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Finish Q-Learning","shortMessageHtmlLink":"Finish Q-Learning"}},{"before":"72f503a7bde09b5a2fe65f4a5365ff579a390fe0","after":"6da5b8eee81322fdaa86452209c13d13e53b9fda","ref":"refs/heads/master","pushedAt":"2023-10-16T12:26:41.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Reorg folder hw2","shortMessageHtmlLink":"Reorg folder hw2"}},{"before":"fd9139426494b890854460dbdb35ba072417ff9c","after":null,"ref":"refs/heads/hw01","pushedAt":"2023-10-14T03:43:09.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"}},{"before":"47e258d768d1ff3e1e3b6b6a31c1270c22404a09","after":"72f503a7bde09b5a2fe65f4a5365ff579a390fe0","ref":"refs/heads/master","pushedAt":"2023-09-28T13:05:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Initialize model free prediction (RL Fall 2023)","shortMessageHtmlLink":"Initialize model free prediction (RL Fall 2023)"}},{"before":"0bbc117715a666741ee76d8ceb27b90c00d4ac72","after":"47e258d768d1ff3e1e3b6b6a31c1270c22404a09","ref":"refs/heads/master","pushedAt":"2023-09-28T11:18:58.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Impl DP solver for MDP","shortMessageHtmlLink":"Impl DP solver for MDP"}},{"before":"50879596385eb336f466bedaa31ec75efd1f59f6","after":"fd9139426494b890854460dbdb35ba072417ff9c","ref":"refs/heads/hw01","pushedAt":"2023-09-27T14:13:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"EdwardLeeMacau","name":"Edward Lee","path":"/EdwardLeeMacau","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/34258587?s=80&v=4"},"commit":{"message":"Update scripts for debugging RL in DP","shortMessageHtmlLink":"Update scripts for debugging RL in DP"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"Y3Vyc29yOnYyOpK7MjAyMy0xMS0yMVQwNjowMjoyOC4wMDAwMDBazwAAAAOz72Dn","startCursor":"Y3Vyc29yOnYyOpK7MjAyMy0xMS0yMVQwNjowMjoyOC4wMDAwMDBazwAAAAOz72Dn","endCursor":"Y3Vyc29yOnYyOpK7MjAyMy0wOS0yN1QxNDoxMzo0NC4wMDAwMDBazwAAAAOKKXzx"}},"title":"Activity · EdwardLeeMacau/reinforcement_learning"}