Home Forums Model Development Kit (MDK) Data preparation for training

This topic contains 4 replies, has 2 voices, and was last updated by  Mostafiz Hossain 3 weeks, 4 days ago.

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
    Posts
  • #15601

    Mostafiz Hossain
    Participant

    Hello there,

    I am getting started with the tensorflow MDK, can you please guide me with the following question.

    1. In the MDK document, it says “By default, the training and validation data in TFRecord format”, but I cannot find any .tfrecord files in the data folder except the face5_label.txt, imagenet_labels.txt and readme.md file. Do we need to create those TFRecord files?

    2. If we have the quantized model(.pb) file for mobilenet or resNet50, from tensorflow can we just convert it to .model without doing the training to do the inference in the GTI chip?

    Regards,

    #15603
    Arpine Soghoyan
    Arpine Soghoyan
    Moderator

    Hi Mostafiz,

    GTI has trained all our reference models (VGG, Resnet18, Resnet50, Mobilenet) on Imagenet Dataset (the models are in SDK/Apps/Models). You can try these models along with image classification applications (Apps/liteDemo, Apps/Demo, Apps/PipelineDemo (on GTI 2803 uses cascading capabilities to run larger models)).
    MDK is used to create your own models from scratch or retrain an existing model. For this, you’ll be using your own dataset. Without retraining, conversion of a regular floating point model into chip supported quantized format will be impossible, as the weights and coefficients have to be adjusted after quantization.

    To convert your dataset into TFRecord format, we recommend the following scripts:

    https://github.com/tensorflow/models/blob/master/research/inception/inception/data/build_image_data.py
    If it’s imagenet, please use – https://github.com/tensorflow/models/blob/master/research/inception/inception/data/build_imagenet_data.py

    An example is shown below:
    python build_image_data.py \
    –train_directory=”train” \
    –train_shards=5 \
    –validation_directory=”val” \
    –validation_shards=5 \
    –num_threads=5 \
    –output_directory=”output” \
    –labels_file=”pic_label.txt”

    Note, that both scripts create an extra background empty class, which needs to be accounted for during the training.

    Hope this helps.
    Arpine

    #15604

    Mostafiz Hossain
    Participant

    Thank you Arpine for your response. Do you have any pre-trained model for mobilenet-SSD or I have to train for mobilenet SSD?

    Also wanted to clarify the model names, correct me if I am wrong.

    1. gti_2803_10bits_test_fc.model == VGG ?
    2. gti_gnet1_fc1000_2803.model == googlenet1?
    3. gti_mnet_fc1000_2803.model == mobilenet v1?
    4. gti_resnet18_2803.model == resnet 18?
    5. gti_resnet50_2803_2chip.model == resnet50

    Regards,
    Mohit

    #15611
    Arpine Soghoyan
    Arpine Soghoyan
    Moderator

    I have a pre-trained model for Mobilenet SSD in Caffe Framework, let me know if you’re interested.

    1. gti_2803_10bits_test_fc.model == VGG ? normally we use 5 bits for activation, but this is an example for using 10 bits activate for VGG type model.
    2. gti_gnet1_fc1000_2803.model == VGG-16 like model slightly modified by GTI, so we named it gnet.
    3. gti_mnet_fc1000_2803.model == mobilenet v1? yes
    4. gti_resnet18_2803.model == resnet 18? yes
    5. gti_resnet50_2803_2chip.model == resnet50

    #15615

    Mostafiz Hossain
    Participant

    Yes can you please send me the caffe model in my email id(m.mehebuba@gmail.com). Do I need to install the caffe MDK to make it run?

Viewing 5 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic.