How Baidu just Paddle platform for open source

Tsung-jen, Huang Xin co-editor

Paddle depth, Baidu today open source learning platform has raised very much developer interest in the field of artificial intelligence, including on-the-job training in Tensorflow and Caffe until some developers. But given the deep learning open source platform is not much, masses are as keen to eat melon as a development priority, just want to know–how about this platform? What others think of this platform? And this platform different from Tensorflow as well as Caffe?

How do ▎ the platform itself

Have existed before the Paddle itself in open source, beginning in 2013, when Baidu depth depth laboratory realized in neural network training, along with computational advertising, text, images, voice training, such as the rapid growth of data, traditional training platform based on single-GPU has been unable to meet demand, led by Xu Wei, the lab built Paddle (Parallel Asynchronous Distributed Deep Learning) how parallel GPU this training platform.

But today, open source simple model of the Paddle is not 3 years ago, Paddle 3 years ago may also be a deep learning platform-independent, not well supported from other platforms, data access needs. But today’s Paddle is already stressed, it features a Spark with the PADDLE is coupled, a depth of heterogeneous distributed system based on Spark. And after and Baidu-related business of “tight friction”, it has iteration two versions: from the Spark on Paddle schema version 1.0, to Spark on PADDLE schema version 2.0. According to the platform open source rules, presumably in Baidu is very handy for internal repairs after a series of bug lab was finally going to Spark on PADDLE and heterogeneous computing platforms are open source. As to why Baidu to raise revenue, and the reason we all know

Deep learning platform, there are many bug–to attract more developers to try and use advanced learning technology, certainly help to improve the grade of Paddle.

▎ Evaluation of outsiders on the platform        

Know Jia Qingyang answer, is now more positive evaluations.

1. very high quality GPU code

2. very good RNN design

3. the design is clean, not too much abstraction, it is much better than TensorFlow. Juicy Couture

4. high speed RDMA part seems to be no open source (possibly because the RDMA for cluster design has certain requirements): Paddle/RDMANetwork.h at master · baidu/Paddle · GitHub

5. design thinking more like first generation DL framework, but the paddle has been for years, also has historical reasons.

5.1 config is hard-code protobuf message, the extension could have an influence.

5.2, you can see many interesting designs similar to the legacy: using the STREAM_DEFAULT macro, and then directed to a non-default TLS way stream:Paddle/hl_base.h at 4fe7d833cf0dd952bfa8af8d5d7772bbcd552c58 · baidu/Paddle · GitHub (Paddle off-the-shelf Mac are not supported? ) Juicy iPhone 5 case

5.3 in the gradient using the traditional coarse-grain forward/backward design (similar to the Caffe). One might say “so paddle does not auto gradient generation”, this is not the way, autograd existence regardless of the size weight and op. In fact, the TensorFlow aware of fine-grained operator super slow speed, gradually back to coarse-grain on the operator. Currently only see here. All in all was a very solid framework, Baidu development skills are good.

Juicy iPhone 5 case

Estimated evaluation of many people read Jia Qingyang, one Baidu data before we post the following engineers perspective CTO Ying Ning’s evaluation

Looked at from design concepts and Caffe is like, but the network model is not as easy to define as Caffe. Maximum contribution is made, distributed, improves the model’s speed. Details will have to look at the code and get started again.

Another with the above two opinions contrast a deep learning scholars

 Tensorflow schema can be thought of as an upgraded version of theano, theano years earlier than the Caffe, but Caffe first train out, and released a successful of convolutional neural network model to get more attention. Relationship between Tensorflow and Caffe fine much, could learn from the Caffe some implementation techniques, essentially nothing. Baidu is likely to be seen after successful implementation of Caffe, much imitated Caffe, while trying to change some things to make it look different from Caffe.

I assume Caffe who directed it, who uses other tools (tensorflow, keras, theano, torch,mxnet) are not into it, say for a few days and then … … Github attention look at it in a month and the amount of code on github to find someone else to write what you know whether he could spray (later no one can see him attend kaggle or other games or in scientific publishing). Now each company released its own deep learning framework (or machine learning framework), such as Microsoft, Amazon, Yahoo, seems to have been no major movement.

▎ Different from the platform with Tensorflow and Caffe

Lei feng’s network (search for “Lei feng’s network”, public interest) applied for a Paddle public beta today, are also being reviewed, although not direct download experience, but the difference between the other two platforms and is not without a trace. Our Caffe before, Tensorflow understand, as well as Paddle data released today.

How Baidu just Paddle platform for open source?

Interfaces audio

Caffe ——cmd, matlab, python

Tensorflow——python, c++

Paddle ——python, c++

(Note: Python is the language developers primarily use only Caffe changes to model internal to use c++. (If there is dissent, welcomes developers to further exchange)

In General

1) Caffe can be said to be the first industrial grade deep learning tool, was founded at the end of 2013 prepared by UC Berkely Jia Yangqing development language a CNN implementation of excellent features, Caffe in computer vision area is still one of the most popular Kit.

Caffe development language C++ and Cuda, very fast, but due to some remnants of historic architecture, its flexibility is not strong enough. And support for recurrent neural networks and language modeling is very poor. Caffe supports all major development system, use a moderate difficulty level.

2) Tensorflow is a second generation Google open source advanced learning technology, RNN API implementation is an ideal, it uses the symbol method for vector operations, develop speed quickly.

Tensorflow better system only supported by various Linux systems and OSX, but its support for the language is fairly comprehensive, and includes Python, C++ and Cuda, developers document is not as comprehensive as Caffe, so use more difficult.

3) and this time Baidu’s Paddle, as the depth of heterogeneous distributed system based on Spark, through the use of GPU and FPGA heterogeneous computing upgrade data-processing capacity per machine, temporarily access to the industry’s “fairly simple, clean design, stable, fast, occupying smaller video memory. “Evaluation of it by using the GPU and FPGA each machine data processing and heterogeneous computing capability has important contacts. But how, still need to wait a few days to observe people experience.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s