I was curious how Claude Opus works to see what I would value in a paid subscription. For awareness, I use Gemini Advanced and GPT Plus already. I don’t have a set experiment, but I do have a lot of research activities, thinking, and information analysis daily. I say this to say this is not a formal analysis of the quality you can find linked below.
However, it is a practical analysis based on my personal experience and expertise of over 30 years.
- YouTube video script reviewing and feedback.
I provided Claude a voice transcript that I am in the final stages of preparing for a voice over. I asked Claude to give me notes. The interface was straightforward, and I thought there was a nice visual cue of the line count 116 lines, even though it was a paste it showed up like a document attachment which also made it easier to read and also showed that it was separating the prompt from the data.
I asked it, “Here is my video script. Can you give me notes on what you think is strong, and what you think I should change?” The response was split into strengths and areas of improvement. The response gave me confidence that the entire script was assessed. It was helpful to see it call out the areas I had been intentional about focusing on, such as a clear problem statement. I asked it to estimate the tokens. It told me, “The video script you provided contains approximately 1185 tokens, based on the standard tokenisation used by language models like GPT-3.” I then asked it to give me suggested edits and “show me those edits in my script so that I can update in my document?” I wanted to see red lines effectively. It came back with the entire script and bold for new text and strike-through for removed. I was able to clearly see the suggestions. I will say that I did not take any of the edits; however, they were reasonable although generic. I would expect with more prompts of the style and specific data points I would want it could do better. What I did like was the readability of the edits. Having to get an update and then do a diff myself or process a lot of bullets of updates is time-consuming. This was much faster.
2. Analysing a set of work that includes documents.
I have an extract of work that I need to analyse and see if Claude can be a companion researcher for me. Can it get the right count of topics? First off, it could not accept a zip file of data. I had to get files 5 at a time which in this case was not too bad because I was looking at about 5 days of data each stored in files. I did give it the four files that had the information of how many items to analyse. It was able to do that successfully. ChatGPT changed the ordering up and did not get the right 13 results; it came up with 8. The prompt was different, and it had to unpack the zip, but I did tell it the specific folder to look at. Can you summarise what occurred for this week? Do a detailed summary for the team and an executive summary for the owner of the company? It has done a good job of taking extracts of files and summarising them. I have confidence based on the results since I had put values in the document that it was not hallucinating. I did not do a personal audit to confirm anything was missing, but I felt nothing was missing. I did get one hallucination of some owners of tasks identified that are not in the group of data being analysed.
3. Proofread my for blog post, created two featured image options , suggested titles, SEO keyphrase, title slug, and Meta Description
I followed up with a revised prompt “Make the blog post changes, also incorporate using british english and generate the image”
I had misunderstood it seemed about Claude Opus generating images.
The featured image prompts worked just so-so on Chat GPT.
4. Miscellaneous
I did have it process a large PDF file over nearly 10 mega bytes and it able to parse quite well and provide a list of information I asked for.
Response time has been fast.
Example Claude Opus reviews.
1. https://youtu.be/ReO2CWBpUYk?si=AEe_BKJiGkxdnXQw