如下所示:
from multiprocessing import Process import os def training_function(...): import keras # 此处需要在子进程中 ... if __name__ == '__main__': p = Process(target=training_function, args=(...,)) p.start()
原文地址:https://stackoverflow.com/questions/42504669/keras-tensorflow-and-multiprocessing-in-python
1、DO NOT LOAD KERAS TO YOUR MAIN ENVIRONMENT. If you want to load Keras / Theano / TensorFlow do it only in the function environment. E.g. don't do this:
import keras def training_function(...): ...
but do the following:
def training_function(...): import keras ...
Run work connected with each model in a separate process: I'm usually creating workers which are making the job (like e.g. training, tuning, scoring) and I'm running them in separate processes. What is nice about it that whole memory used by this process is completely freedwhen your process is done. This helps you with loads of memory problems which you usually come across when you are using multiprocessing or even running multiple models in one process. So this looks e.g. like this:
def _training_worker(train_params): import keras model = obtain_model(train_params) model.fit(train_params) send_message_to_main_process(...) def train_new_model(train_params): training_process = multiprocessing.Process(target=_training_worker, args = train_params) training_process.start() get_message_from_training_process(...) training_process.join()
Different approach is simply preparing different scripts for different model actions. But this may cause memory errors especially when your models are memory consuming. NOTE that due to this reason it's better to make your execution strictly sequential.
以上这篇keras tensorflow 实现在python下多进程运行就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持。
更新日志
- 小骆驼-《草原狼2(蓝光CD)》[原抓WAV+CUE]
- 群星《欢迎来到我身边 电影原声专辑》[320K/MP3][105.02MB]
- 群星《欢迎来到我身边 电影原声专辑》[FLAC/分轨][480.9MB]
- 雷婷《梦里蓝天HQⅡ》 2023头版限量编号低速原抓[WAV+CUE][463M]
- 群星《2024好听新歌42》AI调整音效【WAV分轨】
- 王思雨-《思念陪着鸿雁飞》WAV
- 王思雨《喜马拉雅HQ》头版限量编号[WAV+CUE]
- 李健《无时无刻》[WAV+CUE][590M]
- 陈奕迅《酝酿》[WAV分轨][502M]
- 卓依婷《化蝶》2CD[WAV+CUE][1.1G]
- 群星《吉他王(黑胶CD)》[WAV+CUE]
- 齐秦《穿乐(穿越)》[WAV+CUE]
- 发烧珍品《数位CD音响测试-动向效果(九)》【WAV+CUE】
- 邝美云《邝美云精装歌集》[DSF][1.6G]
- 吕方《爱一回伤一回》[WAV+CUE][454M]