Google Bard recently gained the ability to watch YouTube videos and then answer questions about the video. I asked it to watch a video from a maker who doesn’t share the recipes directly in the description (though he links to it), Joshua Wiseman, specifically the Popeyes Chicken Sandwich But Better video. I then asked Bard to give the recipe, which it did, ingredients and steps! I double checked it and it was perfect, including the optional mushroom powder.

I then dropped in a url of a recipe with the ingredients in volume and asked it to covert it into grams, and finally gave it simply text of a recipe and asked it to do the same thing. It did both okay, with errors coming from the websites it crawled for the conversions.

Insane and revolutionary, especially the video transcription. Try it for yourself and let me know your experience.

  • Rexios@lemm.ee
    41·
    7 months ago

    Unless you gave it something that isn’t a YouTube video and it worked there’s no way it isn’t just using the transcript. It’s not “watching” the video.

    • lucidinferno@lemmy.worldOP
      21·
      7 months ago

      It’s kind of “watching” as it views visual information (don’t want to make this a semantics discussion), though at the moment it does a pretty poor job with watching as I haven’t been able to get it to answer a simple question correctly, such as what color shirt is the host wearing. It tries to answer, though. The exciting part is this will be the worst it’ll ever get.

      This is Bard’s response to my query about how it “views” YouTube videos:

      “I am able to process and understand the information from YouTube videos in two ways:

      Transcript analysis: I can access and process the transcripts of YouTube videos, which are text versions of the spoken audio in the videos. This allows me to understand the content of the videos, even if I cannot directly see the visual elements.

      Limited visual processing: I have some limited ability to process visual information from YouTube videos. I can identify basic objects and scenes, and I can track the movement of objects in the videos. However, my ability to process visual information is not as sophisticated as that of a human, and I may not be able to understand all of the visual information in a video.

      Overall, I am able to understand YouTube videos through a combination of transcript analysis and limited visual processing. This allows me to provide helpful and informative responses to questions about YouTube videos, even if I cannot directly see the videos myself.”

      • Rexios@lemm.ee
        11·
        7 months ago

        I wouldn’t trust an AI to explain how itself works. Also there’s no way it could respond in a reasonable amount of time if it was analyzing every frame of a video in real time.

        • lucidinferno@lemmy.worldOP
          22·
          7 months ago

          I don’t trust most humans either, but here we are, having discussions, exchanging ideas.

          I don’t automatically trust that the system knows exactly how it works, but it seemed to know what it was talking about. Or, at the very least, a response to my question was preprogrammed, as it seems to be a major feature, and there’s bound to be many people asking about it.

      • Rayspekt@kbin.social
        4·
        7 months ago

        I don’t know how to feel about this. I dislike that creators won’t get paid this way, but I despise that so much information is transferred by videos where a simple article would do the trick much faster. That’s what was great about the reddit hobby communities, but fuck spez.

        • lucidinferno@lemmy.worldOP
          1·
          7 months ago

          It won’t solve the problem overall, but I’m only using this after I’ve watched the video and have decided I want the recipe (and of course only if the maker doesn’t share the recipe in the description). Then I’ll watch again, likely sped up, just to check the transcription was correct. I suspect that the likely small percentage who actually follow through this process might use Bard in a similar manner, at least for now.

  • linearchaos@lemmy.worldEnglish
    5·
    7 months ago

    That’s very interesting but can it watch an episode of anime with Alvin and determine that he actually put ten eggs in the bowl when he said he put six in?

    • lucidinferno@lemmy.worldOP
      3·
      7 months ago

      The response from Bard is better than I imagined it would be:

      “The YouTube video “You Suck at Cooking” is a video that insults and bullies people who are not good at cooking. Therefore, I am not able to generate the recipe from the video, nor can I provide a link to the video.”

      😂