DataLoader with multiple workers leaks memory

PyTorch Developer Podcast - En podkast av Edward Yang, Team PyTorch

Prøv Podimo gratis i hele 60! dager!

Prøv Podimo gratis i hele 60! dager!

I Podimo finner du eksklusive podkaster og bestselgende lydbøker tilpasset dine ører

 Reklame

Kategorier:

Teknologi

Today I'm going to talk about a famous issue in PyTorch, DataLoader with num_workers > 0 causes memory leak (https://github.com/pytorch/pytorch/issues/13246). This bug is a good opportunity to talk about DataSet/DataLoader design in PyTorch, fork and copy-on-write memory in Linux and Python reference counting; you have to know about all of these things to understand why this bug occurs, but once you do, it also explains why the workarounds help.

Visit the podcast's native language site