Kontakt
Ako nakupovať
Doručenie
Nákupný poradca
Parallel Computing Cluster for Solving Computational Problems in Data
A Framework for High-Throughput Data Processing. DE
Angličtina
95 b
30 dní na vrátenie tovaru
Thisbook tackles a critical bottleneck in large-scale AI: the slow and communication-heavy training of massive Deep Neural Networks (DNNs) on multi-GPU systems. It addresses the trade-off between two main parallelization methods. Data parallelism suffers from severe communication overhead for large models, while pipelined model parallelism (like PipeDream) offers up to 8.91x speedup for large Fully Connected/Recurrent Neural Networks but causes "weight staleness," degrading model accuracy. To resolve this, the paper introduces SpecTrain, a novel technique. SpecTrain uses the momentum from optimizers to predict future weight updates, allowing pipelined computation with accurate, non-stale weights. This enables the high GPU utilization and speed of pipelining while maintaining the training robustness and final accuracy of synchronous methods.
Informácie o knihe
Angličtina
Darujte túto knihu ešte dnes
Je to jednoduché
1 Pridajte knihu do košíka a vyberte možnosť doručiť ako darček 2 Obratom Vám zašleme poukaz 3 Knihu zašleme na adresu obdarovanéhoAhoj! Som Libroamiko, tvoj knižný radca.
Ako ti môžem pomôcť?