I was trying to make my BBQ cook recipes with machine for a week. Here is the experience what I learned from trying to implement it.

First Lesson: Always test training process with small circle before running entire session.

Test your script with 1 epoch. It can save your 30 hours work without sigh.

Let us take a look at code snippet after sigh.

If you follow the tutorial to train a LSTM, you will copy below command and change “my_own_data” in train.py file.

However, it will return an empty string.

Second Lesson: Run your training program on a GPU machine if possible.

Machine learning is really time consuming.

and TensorFlow do not support GPU mode on Mac OS.

This training was crushed on my laptop. The actual steps I need is 1,320,000.

Koji helped me run my training in NYU HPC machine. However, the training still got crashed after one day for uncertain issue. sigh

Third Lesson: Never train your data during the finals on a shared machine.

or it will crush with Unreal…..

sigh…

Fourth Lesson: Checkpoint is my lifesaver.

Finally, I gave up training my own BBQ receipts data and feed myself TensorFlow documents with learned helplessness. Wait, what is Checkpoint?

It seems possible to restart training process. However, I did not want to continue my data training anymore. But, I still need to show something for my NOC final. Could I use the unfinished training data to generator a model?