I’ve been attempting to convert a Caffe model to evaluate the Plai Plug 2803 following the MDK User Guide, and I am quite stuck. Our model mostly consists of Convolution and ReLU layers, as well as some Concat, Deconvolution, and Eltwise layers. It has already been trained on vanilla Caffe (i.e., without GTI modifications).
The User Guide, page 24, says “After performing steps described in section 7. Model Development Workflow, the layers should be “QuantConvolution” layers and activations should be “QuantReLu” layers.” Is this trying to say all Convolutions should be QuantConvolutions and all ReLUs should be QuantReLUs? If so, how was this supposed to happen? Steps 7.2-7.4 are “recommended”, and Step 7.1 says to train a model, which I’ve already done. Am I supposed to just edit my prototxt, replacing Convolution/ReLU layers with QuantConvolution/QuantReLU, respectfully? Do I need to retrain with your Caffe modifications?
My next question regards the network*.json and Fullmodel_def *.json arguments to convert.py. How exactly am I supposed to create these? I find the User Guide unhelpful here: “network*.json is a JSON file similar to network _*_template.json provided in the conversion_tool/network_examples directory” (and similarly for the Fullmodel_def*.json). To me, those network_example jsons are a bit complicated, and I’m left wondering how to create my own (or even why they are necessary, given the prototxt and caffemodel). Appendix A : JSON File Format does not show me how to set them either.
GTI offers model development kit (MDK) to create your own models either from scratch or using pretrained caffemodel, train and convert to a format that will fit into GTI chip (.model format). MDK includes examples of network definitions for Mobilenet, Resnet and VGG type networks (for GTI 2803) that can be used as a starting point for your development. We recommend first running the training based on these examples than modifying them to match the required network criteria. If you have a custom network, please first check our model specification guide to make sure that particular network structure fits GTI chip architecture (GTI 2801 or GTI 2803).
the steps to run the MDK.
1) Run the training with full floating point model.
2) Run the training with quantized convolutional layers.
3a) Calibrate the range of activation values.
3b) Run the training with quantized activation layers.
4) Apply model fusion.
5) Evaluate the quantized and fused model on the CPU/GPU.
6) Convert the caffemodel into .model (GTI chip format).
7) Run inferencing on the GTI chip. The accuracy at this stage should be the same as the accuracy in step 1 (~2% variation is possible).
In my steps this is what I have done.
1) I used the example prototxt to run training from scratch.
2) Then replace FLOATING_POINT with
quant_convolution_param {
coef_precision: THREE_BIT
bw_params: 12
#shift_enable: true
shift_enable: false
}
Make sure quant_enable: false
3a) Run Calibrate_QuantReLU.py.
3b) Then turn on the quantized activation layers
By setting quant_enable: true
5) Run RefineNetwork.py
Make sure to set image_means=[0,0,0] otherwise default values will be used.
6) Create deploy prototxt based on trainprototxt
7) Run the conversion tool.
If you need help with custom network training, GTI offers reviewing it. Send a DM with your network for me to review.
Thank you for your response, but unfortunately it doesn’t answer my questions:
I have a pretrained model from vanilla caffe (not GTI caffe) that I do not want to retrain, but want to run on a GTI device. How do I get a corresponding prototxt and caffemodel for this model with GTI QuantConv and QuantReLU layers? Then, once I have such a model, how do I get the network*.json and Fullmodel_def*.json for my model?. My original question includes further details on my current understanding.
Retraining is mandatory, as GTI chip requires a specific model format to run on the chip.
As I have mentioned, the MDK already includes example network.json and fullmodel.json files for your reference, however if you’re having issues creating your own custom network structure. please share more information so I can guide you properly.
Just to confirm, the MDK User Guide for Caffe page 33 says “The MDK has options to train the model on [sic] from a pretrained model or from scratch on CPU or GPU” but you’re saying I must retrain? I have a pretrained model which I’m not interested in retraining (it takes days).
Regarding the network.json and fullmodel.json, I am already aware of what the user guide says on them (as my first post said). My problem is I do not understand how to generalize them to my own network. The guide only says “network*.json is a JSON file similar to network _*_template.json provided in the conversion_tool/network_examples directory.” How is one supposed to create their own? There are many parameters present in your example JSON files.
I understand that it would be so much easier simply porting your existing model directly into the chip, however, without retraining your pretrained model with GTI MDK it wouldn’t be possible to deploy it into the chip.
If you need help customizing network*.json and full_model*.json files, I already offered reviewing your network prototxt.
Regards,
Arpine
Author
Posts
Viewing 6 posts - 1 through 6 (of 6 total)
You must be logged in to reply to this topic.
We use cookies to offer you a better browsing experience and analyze site traffic. By using our site, you consent to our use of cookies. AcceptRead our Cookies Policy
GTI Terms of Service for Self-Service Developer Portal
Last updated: November 13, 2018.
Welcome to the Gyrfalcon Technology Inc. (“GTI”) Self-Service Developer Portal (“Dev Portal”). While we try to write the GTI Terms of Service (“Terms”) in an easy to understand format, your access to and use of the Dev Portal is conditioned on your acceptance of and compliance with the Terms. By using the Dev Portal and related materials made available to you, you agree to be bound by the Terms.
Protect Your Account – You are provided with access to a password-protected Dev Portal. You will protect the login credentials provided and not share them with any third parties. You are responsible for maintaining the confidentiality of the information you submit through your account and the corresponding password, and for restricting access to your account. You agree to accept responsibility for all activities that occur under your account or password.
Confidentiality – All information posted on the Dev Portal is confidential. You agree to not share any information made available on the Dev Portal publicly or with third parties not associated with the Dev Portal. You shall not (nor allow, authorize or assist others to): decompile, reverse engineer, disassemble, modify, sublicense, network, rent, lease, loan, timeshare, sell, transmit, distribute, disclose, publicly display, publish, reproduce, create derivative works based on, assign or transfer the materials or information in relation to the Dev Portal including Feedback (as defined below) to any other person or entity.
Ownership – GTI owns the Dev Portal, accompanying documentation and services, including all related intellectual property rights therein. All of our rights not expressly granted by these terms are hereby retained.
Feedback – You may provide GTI with comments concerning the Dev Portal or your evaluation and use thereof (“Feedback”). Your Feedback is vital to the growth and development of the Dev Portal. We want to know what is valuable to you, and what additional resources you would like GTI to provide. You agree that the Feedback is the exclusive property and confidential information of GTI, and you hereby assign all Feedback to GTI at no cost. GTI may freely use, reproduce, license, distribute, and otherwise commercialize Feedback without the payment of any royalties or fees, but is under no obligation to use or incorporate Feedback in any way.
No Warranties – All information, software or resources are provided on an “as is” and “as available” basis without any warranties of any kind, and we expressly disclaim any and all warranties, whether express or implied, including the implied warranties of merchantability, title, fitness for a particular purpose, and non-infringement. You acknowledge that we do not warranty that the Dev Portal will be uninterrupted, timely, secure, or error-fee.
Limitation of Liability – In no event will GTI’s aggregate liability arising out of or related to the Terms or your access to the Dev Portal (whether in contract or tort or under any other theory of liability) exceed US$500. In no event will GTI have any liability to you or any third party for any lost profits or revenues or for any indirect, special, incidental, consequential, cover or punitive damages however caused, whether in contract, tort or under any other theory of liability, and whether or not you or the third party has been advised of the possibility of such damage. The foregoing disclaimer will not apply to the extent prohibited by applicable law.
Indemnity– You will indemnify, defend, and hold GTI and its employees, directors, agents, affiliates and representatives harmless from and against any and all claims, costs, losses, damages, judgments, tax assessments, penalties, interest, and expenses (including without limitation reasonable attorneys’ fees) arising out of any claim, action, audit, investigation, inquiry, or other proceeding instituted by a person or entity that arises out of or relates to: (a) any actual or alleged breach of your obligations set forth in these Terms; (b) your wrongful or improper use of the Dev Portal; (c) your violation of any third-party right, including without limitation any right of privacy, publicity rights or Intellectual Property Rights; (d) your violation of any law, rule or regulation of the United States or any other country; or (e) any other party’s access and/or use of the Dev Portal with your unique name, password or other appropriate security code.
Links To Other Web Sites – The Dev Portal may contain links to third-party web sites or services that are not owned or controlled by GTI. GTI has no control over, and assumes no responsibility for, the content, privacy policies, or practices of any third party web sites or services. You further acknowledge and agree that GTI shall not be responsible or liable, directly or indirectly, for any damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods or services available on or through any such web sites or services.
Termination – We may terminate or suspend access to the Dev Portal immediately, without prior notice or liability, without limitation, if you breach the Terms. All provisions of the Terms which by their nature should survive termination shall survive termination, including, without limitation, ownership provisions, warranty disclaimers, indemnity and limitations of liability.
Changes – GTI reserves the right, at our sole discretion, to modify or replace the Terms at any time. If a revision is material we will make reasonable efforts to notify you prior to new terms taking effect. What constitutes a material change will be determined at our sole discretion.
Governing Law – The Terms and claims arising under the Terms are governed by the laws of California without regard to its choice of law or conflicts of law principles. The state and/or federal courts residing in Santa Clara County, California shall have exclusive jurisdiction over any dispute or claim arising out of the Terms. You may not export the information and materials included in or relating to the Dev Portal in violation of applicable export laws and regulations.
Contact Us – If you have any questions about these Terms, please contact us at dev@gyrfalcontech.ai.
By agreeing to these Terms of Service, you also agree to have read and accept our End User License Agreement. Please read this document before proceeding.