Image captioning for Spanish with pre-trained vision and text model For this project, a pre-trained image model like ViT can be used as an encoder, and a pre-trained text model like BERT and/or GPT2 can be used as a decoder. Model Pre-trained ViT , BERT models can be found on the model hub. We could…

Hi, I would like to join this project. I am not experienced enough with NLP to create a project on my own. But I do have experience with Computer Vision overall, and would like to try and see how well spanish text is generated for the image-captioning. I am David and I am in the Amsterdam timezon…

So you want to create a CLIP like (dual encoder) model? @valhalla

Hey @dmatos2012 , don’t worry about experience. We always try to make things easier for everyone and we have a super cool speaker lineup for getting familiar with JAX/Flax/Transformers. And we will try to answer all questions:) @mrm8488 For image captioning it’ll be more like an encoder-decoder mod…

Hey there, this is very interesting, I have some experience with NLP and computer vision, and always wanted to get more experience with multi-modal models (text + vision), also since I saw the WIT dataset for the first time, I wanted to use it for some project, this seems a good opportunity. If you…

Great, let’s define the project! @mrm8488 let me know if you want to be added here as well :slight_smile:

Hello @valhalla @patrickvonplaten , Hope you are doing well… I am interested in this project as well… So much to learn from doing this project…It is similar to CLIP. But in Clip, we use encoders for both image & text… In this project, we are gonna use encoder for image & decoder for text. Also, I…

Hey @srisweet ! Added you the team :slight_smile: Great idea, feel free to create a thread to post JAX/Flax resources!

Hi @valhalla . could you add me as well please? I replied at the top of the post :). Thank you!

hi @dmatos2012 you are already part of the team :slight_smile: Here’s the sheet

Image captioning for Spanish with pre-trained vision and text model

Flax/JAX Projects

srisweet June 30, 2021, 9:54am 13

Hi @valhalla ,
Just created this thread with JAX/Flax resources…

Topic		Replies	Views
Image captioning for French with pre-trained vision and text model Flax/JAX Projects	6	2255	January 4, 2022
Image captioning for Japanese with pre-trained vision and text model Flax/JAX Projects	0	1222	June 23, 2021
CLIP like contrastive vision-language models for Spanish with pre-trained text and vision models Flax/JAX Projects	4	439	June 29, 2021
Multilingual Image Captioning Flax/JAX Projects	10	1408	July 6, 2021
Image captioning for Indonesia with pre-trained vision Flax/JAX Projects	4	565	June 29, 2021

Image captioning for Spanish with pre-trained vision and text model

Related topics