• 1 Post
  • 404 Comments
Joined 3 years ago
cake
Cake day: July 5th, 2023

help-circle





  • The only solution is to make sure they can’t read data you don’t want shared.

    Isn’t that the appropriate guardrail, then? LLM chats and agents and whatever need to be contained with external permissions settings that the LLMs simply do not and can never have the power to override.

    In a normal customer service setting with human agents, there are still plenty of examples of what a human agent simply doesn’t have the power to do. Often, they’ll need to escalate to a manager to do things like process refunds not just because they weren’t given social permission to do so, but because they weren’t given technical permissions to do so. LLM agents need to be contained in the same way. Any decent use of agents, human or software, requires carefully designed processes and permissions extrinsic to that agent’s own decisionmaking abilities to make sure that agents don’t do something bad for the company.





  • Yeah, the smarter way to use LLM-based agents is carefully defined tasks. Mozilla describes their vulnerability assessment processes in this blog post.

    Mozilla describes the process they’ve used: building a harness that instructs a model to find a specific category of vulnerability on a specific interface, and then write up its findings. It’s a narrow enough context that the model gets specific instructions, and a simple definition of success, and it sets up many such tasks that can be fed into the existing process for verifying and triaging bugs. Note that the output for this LLM pipeline basically feeds into the same interface for accepting bug reports from the public, or from their human contributors within the project.

    There’s a couple of takeaways here, too:

    • This pipeline is model agnostic. Mozilla set it up before Mythos was released, and its description of other models (Opus 4.7, Codex) confirms that Mythos is better but not a true game changer. The ability to swap out other models provides some assurance that the work done to develop the pipeline will be useful when cheaper or better models come along, or when a model becomes unavailable (like when a provider decides a particular model is too expensive to run, or a provider goes under).
    • The increase in automated output (and presumably automation-assisted contributions from the public) has given the humans more work to do. Automation in this context actually increases the demand for human labor.
    • Other projects will need to develop their own custom pipelines, specific to their project, to get good results from LLM based agents.

    There are ways to use these tools, but none of it really seems like a truly revolutionary/disruptive change to how large projects are managed.







  • AI has an interesting economic trait in that it’s very, very expensive to deploy, and made very fast progress from 2022 to 2024. That caused investors with money to believe that:

    • Pushing the frontier was going to cost a lot of money. More than any other purported revolutionary tech.
    • Extrapolation of past improvement meant that whoever was on the cutting edge may end up with a product with a huge paying market.
    • So whoever wins this race would be rich, and the investment would have been worth it for them.

    But since 2024, we’ve seen that the cutting edge got even more expensive much faster than expected, and much of the improvements in performance now come from inference rather than training, which represents a high ongoing cost.

    Now, if we extrapolate from that trend line, we’ll see that the market will be much smaller for AI services at the cost it takes to provide that service, and the question then becomes whether the industry can make its operations cheaper, fast enough to profitably provide a service people will pay for.

    I have my doubts they’ll succeed, and we might just be looking at the industry like supersonic flight: conceptually interesting, technically feasible, but just a commercial dead end because it’s too expensive.


  • ActivityPub is the fediverse protocol, lemmy and piefed are software implementations of that protocol.

    In a similar way, email is a protocol, and Gmail and Exchange/Outlook are software implementations of that protocol. You can use Gmail to send email to Outlook users. Different people can administer their own Exchange servers on their own domains.

    And some features of the software work best with other people who use the same version of the same software, although most things kinda work between different software. Like how calendar invites sometimes act weird between users of different software, but for the most part the core functions work OK.

    So lemmy and piefed (and mbin and sublinks and some others) are different software trying to speak to other fediverse services through the ActivityPub protocol. It mostly works, but some of the details don’t work exactly the same between each type of software.


  • One real concern I have is that there are now automated tools that can read a patch, and the maintainer’s release notes with a description of a security vulnerability fixed by that patch, and then create a working exploit of the pre-patch vulnerability.

    In that particular moment, you know that a vulnerability exists and that it was serious enough to be described in release notes, and you can compare two code versions, one that is secure and one that is not. From there, any AI coding agent is working towards something that definitely exists, with a bunch of description of what it might be.

    So that means that the window between when a patch is released and when users actually apply that patch is going to be more important than ever. Downstream maintainers will be under a lot of time pressure to implement changes from upstream, because every new security patch will create a race to create 1-day exploits for everyone using that software.