Vysper cluster, load balance, and LVS

 

 http://www.mail-archive.com/[email protected]/msg04084.html

http://comments.gmane.org/gmane.comp.apache.mina.user/3665

 

HI, all,

We have a requirement which needs clustering in vysper.I came acrossthis link (http://comments.gmane.org/gmane.comp.apache.mina.user/3665)where this is discussed using cassandra.Is this implemented in vysper.Trying to implement this by RosterManager.Need a clarification duringclustering what needs to stored is it the resource or session boundwith resource to share the users object among multiple vysper servers

 

clustering in vysper aruna reddy

• Re: clustering in vysper Bernd Fondermann
  • Re: clustering in vysper Sasha Dolgy
    • Re: clustering in vysper aruna reddy
      • Re: clustering in vysper Sasha Dolgy
        • Re: clustering in vysper aruna reddy
      • Re: clustering in vysper Bernd Fondermann
        • Re: clustering in vysper aruna reddy
          • Re: clustering in vysper Bernd Fondermann

 

 

Hi everyone,

Been doing some research into XMPP compliant servers and happily stumbled
upon Vysper and MINA which I've had great success with in the past.  So, a
few questions:

1.  is it possible to deploy multiple instances of vysper to multiple
servers and they are aware of each other?  so that i can introduce a
solution that has no points of failure...
2.  i want to leverage apache cassandra as a backend repository for data.
to do this, i would create a storage provider that handles the cassandra
interaction?

thanks in advance
-sasha

--

-- 
Sasha Dolgy
sasha.dolgy@...

Permalink | Reply |

headers

Niklas Gustavsson | 18 Feb 11:00

Re: [vysper] questions

Niklas Gustavsson < niklas@...>
2011-02-18 10:00:51 GMT

On Fri, Feb 18, 2011 at 10:51 AM, Sasha Dolgy <sdolgy@...> wrote:
> Been doing some research into XMPP compliant servers and happily stumbled
> upon Vysper and MINA which I've had great success with in the past.

That's good to hear 

> 1.  is it possible to deploy multiple instances of vysper to multiple
> servers and they are aware of each other?  so that i can introduce a
> solution that has no points of failure...

No, we do not currently support clustering of Vysper. However, this is
an area that I'm personally is interested in looking into so I would
be happy to collaborate on this if you like.

> 2.  i want to leverage apache cassandra as a backend repository for data.
> to do this, i would create a storage provider that handles the cassandra
> interaction?

Yes.

/niklas

Permalink | Reply |

headers

Sasha Dolgy | 18 Feb 11:12

Re: [vysper] questions

Sasha Dolgy < sdolgy@...>
2011-02-18 10:12:39 GMT

Hi Niklas,

The more I think about it, and the more I read, the vysper instances may not
need to be aware of each other ... this can be accomplished by all vysper
instances using a common storage provider.  Similar to how we cluster http
servers ontop of a shared storage pool.

It shifts the complexity down a level and makes mass deployment of new
instances much easier and doesn't overly complicate the current
implementation.

If I make headway creating a storage provider interfacing with cassandra
i'll drop a note here.

-sd

On Fri, Feb 18, 2011 at 11:00 AM, Niklas Gustavsson <niklas@...>wrote:

>
> > 1.  is it possible to deploy multiple instances of vysper to multiple
> > servers and they are aware of each other?  so that i can introduce a
> > solution that has no points of failure...
>
> No, we do not currently support clustering of Vysper. However, this is
> an area that I'm personally is interested in looking into so I would
> be happy to collaborate on this if you like.
>
> > 2.  i want to leverage apache cassandra as a backend repository for data.
> > to do this, i would create a storage provider that handles the cassandra
> > interaction?

(Continue reading)

Permalink | Reply |

headers

Bernd Fondermann | 18 Feb 12:06

Re: [vysper] questions

Bernd Fondermann < bernd.fondermann@...>
2011-02-18 11:06:12 GMT

On Fri, Feb 18, 2011 at 11:12, Sasha Dolgy <sdolgy@...> wrote:
> Hi Niklas,
>
> The more I think about it, and the more I read, the vysper instances may not
> need to be aware of each other ... this can be accomplished by all vysper
> instances using a common storage provider.  Similar to how we cluster http
> servers ontop of a shared storage pool.

To achieve this we would need to push even more state to storage
providers than we do now.
Hint: org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry
must be shared on all instances.

> It shifts the complexity down a level and makes mass deployment of new
> instances much easier and doesn't overly complicate the current
> implementation.
>
> If I make headway creating a storage provider interfacing with cassandra
> i'll drop a note here.

By all means, please do that! I think cassandra is the perfect backend
for Vysper.
If you'd contribute code, that would be awesome (I don't remember
having used this word for some time.).

  Bernd

Permalink | Reply |

headers

Sasha Dolgy | 18 Feb 13:15

Re: [vysper] questions

Sasha Dolgy < sdolgy@...>
2011-02-18 12:15:37 GMT

Hi Bernd,

I had a look at
org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the way
Vysper currently works today is that all sessions are stored in memory?  If
vysper instance crashes, the resource id : session relationships aren't
persisted anywhere?

It's been years since I properly programmed .. Have only gotten involved in
Cassandra in the past four weeks and have made some significant mistakes and
successes along the way.  For me, I am curious to deploy a compliant XMPP
solution that potentially will leverage the user information i persist in
cassandra.  i started work on writing a module in my application for user
interactions (online / offline messages) only to stop suddenly late last
night and ask myself why i'm re-inventing a wheel ....

-sd
On Fri, Feb 18, 2011 at 12:06 PM, Bernd Fondermann <
bernd.fondermann@...> wrote:

> On Fri, Feb 18, 2011 at 11:12, Sasha Dolgy <sdolgy@...> wrote:
> > Hi Niklas,
> >
> > The more I think about it, and the more I read, the vysper instances may
> not
> > need to be aware of each other ... this can be accomplished by all vysper
> > instances using a common storage provider.  Similar to how we cluster
> http
> > servers ontop of a shared storage pool.
>

(Continue reading)

Permalink | Reply |

headers

Bernd Fondermann | 18 Feb 13:29

Re: [vysper] questions

Bernd Fondermann < bernd.fondermann@...>
2011-02-18 12:29:07 GMT

On Fri, Feb 18, 2011 at 13:15, Sasha Dolgy <sdolgy@...> wrote:
> Hi Bernd,
>
> I had a look at
> org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the way
> Vysper currently works today is that all sessions are stored in memory?  If
> vysper instance crashes, the resource id : session relationships aren't
> persisted anywhere?

No they aren't. But since XMPP uses long-living TCP/IP connections,
recovery from a server crash is equivalent to a
client/server-reconnect anyway.
This does not impose practical problems. (Leaving a reverse proxy setup aside.)
However, a cluster of XMPP servers for the exact same domain(!) would
need to share the ResourceRegistry datastructure.
(You could also have a federation of XMPP servers, but that'd mean
different users might have different account names like
you@..., me@...)

> It's been years since I properly programmed .. Have only gotten involved in
> Cassandra in the past four weeks and have made some significant mistakes and
> successes along the way.  For me, I am curious to deploy a compliant XMPP
> solution that potentially will leverage the user information i persist in
> cassandra.  i started work on writing a module in my application for user
> interactions (online / offline messages) only to stop suddenly late last
> night and ask myself why i'm re-inventing a wheel ....

Vysper can easily be embedded into other Java apps.

  Bernd

(Continue reading)

Permalink | Reply |

headers

Sasha Dolgy | 18 Feb 14:22

Re: [vysper] questions

Sasha Dolgy < sdolgy@...>
2011-02-18 13:22:55 GMT

One more clarification.  The JCR work in Vysper
(org.apache.vysper.storage.jcr) is to use JCR as a storage provider for user
management, roster management, etc.  ?

On Fri, Feb 18, 2011 at 1:29 PM, Bernd Fondermann <
bernd.fondermann@...> wrote:

> On Fri, Feb 18, 2011 at 13:15, Sasha Dolgy <sdolgy@...> wrote:
> > Hi Bernd,
> >
> > I had a look at
> > org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the
> way
> > Vysper currently works today is that all sessions are stored in memory?
>  If
> > vysper instance crashes, the resource id : session relationships aren't
> > persisted anywhere?
>
> No they aren't. But since XMPP uses long-living TCP/IP connections,
> recovery from a server crash is equivalent to a
> client/server-reconnect anyway.
> This does not impose practical problems. (Leaving a reverse proxy setup
> aside.)
> However, a cluster of XMPP servers for the exact same domain(!) would
> need to share the ResourceRegistry datastructure.
> (You could also have a federation of XMPP servers, but that'd mean
> different users might have different account names like
> you@..., me@...)
>

(Continue reading)

Permalink | Reply |

headers

Bernd Fondermann | 18 Feb 14:34

Re: [vysper] questions

Bernd Fondermann < bernd.fondermann@...> 2011-02-18 13:34:10 GMT

On Fri, Feb 18, 2011 at 14:22, Sasha Dolgy <sdolgy@...> wrote:
> One more clarification.  The JCR work in Vysper
> (org.apache.vysper.storage.jcr) is to use JCR as a storage provider for user
> management, roster management, etc.  ?

yep. but I'm not very happy with them.

  Bernd

>
> On Fri, Feb 18, 2011 at 1:29 PM, Bernd Fondermann <
> bernd.fondermann@...> wrote:
>
>> On Fri, Feb 18, 2011 at 13:15, Sasha Dolgy <sdolgy@...> wrote:
>> > Hi Bernd,
>> >
>> > I had a look at
>> > org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the
>> way
>> > Vysper currently works today is that all sessions are stored in memory?
>>  If
>> > vysper instance crashes, the resource id : session relationships aren't
>> > persisted anywhere?
>>
>> No they aren't. But since XMPP uses long-living TCP/IP connections,
>> recovery from a server crash is equivalent to a
>> client/server-reconnect anyway.
>> This does not impose practical problems. (Leaving a reverse proxy setup
>> aside.)
>> However, a cluster of XMPP servers for the exact same domain(!) would

(Continue reading)

Permalink | Reply |

headers

Sasha Dolgy | 18 Feb 16:35

Re: [vysper] questions

Sasha Dolgy < sdolgy@...>
2011-02-18 15:35:53 GMT

Hi,

 I have a working prototype using cassandra as a storage provider for
[ org.apache.vysper.xmpp.authorization.AccountManagement/UserAuthorization
]

verifyAccountExists(Entity jid)
addUser(String username, String password)

I started taking a look at the ResourceRegistry as you suggested.  Wouldn't
a good first step be to abstract this and make it an interface and move the
existing code into something that then implements it?  This way people could
write different resource registry implementations so that we could see just
how badly having it back onto Cassandra would be ; )

My code isn't pretty.  Having said that, what or where is the best way to
share it?  Admittedly, it has been years since i've contributed something
anywhere.

Also, what I find a little odd, the password for an entity is in clear
text....?  Is this covered in the XMPP protocols anywhere?

-sd

On Fri, Feb 18, 2011 at 12:06 PM, Bernd Fondermann <
bernd.fondermann@...> wrote:

>  > It shifts the complexity down a level and makes mass deployment of new
> > instances much easier and doesn't overly complicate the current
> > implementation.

(Continue reading)

Permalink | Reply |

headers

Bernd Fondermann | 18 Feb 16:43

Re: [vysper] questions

Bernd Fondermann < bernd.fondermann@...>
2011-02-18 15:43:27 GMT

On Fri, Feb 18, 2011 at 16:35, Sasha Dolgy <sdolgy@...> wrote:
> Hi,
>
>  I have a working prototype using cassandra as a storage provider for
> [ org.apache.vysper.xmpp.authorization.AccountManagement/UserAuthorization
> ]
>
> verifyAccountExists(Entity jid)
> addUser(String username, String password)
>
> I started taking a look at the ResourceRegistry as you suggested.  Wouldn't
> a good first step be to abstract this and make it an interface and move the
> existing code into something that then implements it?

+1

> This way people could
> write different resource registry implementations so that we could see just
> how badly having it back onto Cassandra would be ; )

right 

> My code isn't pretty.  Having said that, what or where is the best way to
> share it?  Admittedly, it has been years since i've contributed something
> anywhere.

Please open a JIRA ticket (
https://issues.apache.org/jira/browse/VYSPER , registration required)
create a diff (IDEs, Unix, SVN tools etc support that) and attach it
to the JIRA issue.

(Continue reading)

Permalink | Reply |

headers

Sasha Dolgy | 18 Feb 16:59

Re: [vysper] questions

Sasha Dolgy < sdolgy@...>
2011-02-18 15:59:39 GMT

On Fri, Feb 18, 2011 at 4:43 PM, Bernd Fondermann <
bernd.fondermann@...> wrote:

>
> > I started taking a look at the ResourceRegistry as you suggested.
>  Wouldn't
> > a good first step be to abstract this and make it an interface and move
> the
> > existing code into something that then implements it?
>
> +1
>
> > This way people could
> > write different resource registry implementations so that we could see
> just
> > how badly having it back onto Cassandra would be ; )
>
> right 
>

https://issues.apache.org/jira/browse/VYSPER-266

>
>
> > Also, what I find a little odd, the password for an entity is in clear
> > text....?  Is this covered in the XMPP protocols anywhere?
>
> Ah. Next JIRA, please. We shouldn't do that, and no storage backend is
> required to store plain text.
> XMPP is agnostic about how passwords are stored.

(Continue reading)

Permalink | Reply |

headers

Sasha Dolgy | 18 Feb 19:49

Re: [vysper] questions

Sasha Dolgy < sdolgy@...>
2011-02-18 18:49:01 GMT

> https://issues.apache.org/jira/browse/VYSPER-266
>

uploaded a patch for this.
https://issues.apache.org/jira/secure/attachment/12471431/ResourceRegistry-to-Interface.patch

No doubt the patch could be cleaned up.  Hoping a few others may see some
value in this.  I for one would like the ability to create an alternate
method for the ResourceRegistry like I can with Storage providers which is
why i've done this without affecting the default approach.

Thanks for the mini-tutorial Bernd about getting up to speed and helping
out.   Appreciate it.

-sd

Permalink | Reply |

headers

Niklas Gustavsson | 18 Feb 15:48

Re: [vysper] questions

Niklas Gustavsson < niklas@...>
2011-02-18 14:48:19 GMT

On Fri, Feb 18, 2011 at 11:12 AM, Sasha Dolgy <sdolgy@...> wrote:
> The more I think about it, and the more I read, the vysper instances may not
> need to be aware of each other ... this can be accomplished by all vysper
> instances using a common storage provider.  Similar to how we cluster http
> servers ontop of a shared storage pool.

I don't think this is sufficient since we do not currently support
relaying messages between users connected to the same domain but
different servers.

/niklas

Permalink | Reply |

headers

Sasha Dolgy | 18 Feb 16:40

Re: [vysper] questions

Sasha Dolgy < sdolgy@...>
2011-02-18 15:40:06 GMT

Hi Niklas,

Although vysper doesn't currently support it, do the XMPP standards support
it?
-sd

On Fri, Feb 18, 2011 at 3:48 PM, Niklas Gustavsson <niklas@...>wrote:

> I don't think this is sufficient since we do not currently support
> relaying messages between users connected to the same domain but
> different servers.
>
> /niklas
>

Permalink | Reply |

headers

Bernd Fondermann | 18 Feb 16:48

Re: [vysper] questions

Bernd Fondermann < bernd.fondermann@...>
2011-02-18 15:48:12 GMT

On Fri, Feb 18, 2011 at 16:40, Sasha Dolgy <sdolgy@...> wrote:
> Hi Niklas,
>
> Although vysper doesn't currently support it, do the XMPP standards support
> it?
> -sd

I assume you're referring to the clustering...
No, XMPP does not provide an extension yet. There is one[1], but it is
"deferred" and not recommended for implementation.
The XSF ( xmpp.org ) is a very open-minded community. If you'd want to
go that far you could start a new spec effort there.

   Bernd

[1] http://xmpp.org/extensions/xep-0051.html

>
>
> On Fri, Feb 18, 2011 at 3:48 PM, Niklas Gustavsson <niklas@...>wrote:
>
>> I don't think this is sufficient since we do not currently support
>> relaying messages between users connected to the same domain but
>> different servers.
>>
>> /niklas
>>
>

(Continue reading)

Permalink | Reply |

headers

Niklas Gustavsson | 18 Feb 16:50

Re: [vysper] questions

Niklas Gustavsson < niklas@...> 2011-02-18 15:50:12 GMT

On Fri, Feb 18, 2011 at 4:40 PM, Sasha Dolgy <sdolgy@...> wrote:
> Although vysper doesn't currently support it, do the XMPP standards support
> it?

The XMPP RFCs specifies the hostname lookup, using DNS SRV records.
But beyond that, I don't think there is any active spec for how to
cluster-enable XMPP servers.

/niklas

Permalink | Reply |

headers

Ashish | 18 Feb 11:51

Re: [vysper] questions

Ashish < paliwalashish@...>
2011-02-18 10:51:10 GMT

On Fri, Feb 18, 2011 at 3:30 PM, Niklas Gustavsson <niklas@...> wrote:
> On Fri, Feb 18, 2011 at 10:51 AM, Sasha Dolgy <sdolgy@...> wrote:
>> Been doing some research into XMPP compliant servers and happily stumbled
>> upon Vysper and MINA which I've had great success with in the past.
>
> That's good to hear 
>
>> 1.  is it possible to deploy multiple instances of vysper to multiple
>> servers and they are aware of each other?  so that i can introduce a
>> solution that has no points of failure...
>
> No, we do not currently support clustering of Vysper. However, this is
> an area that I'm personally is interested in looking into so I would
> be happy to collaborate on this if you like.

Would like to hear more on what all we need to cluster :)
I might be able to help out here a bit, as I am working on something similar.

>
>> 2.  i want to leverage apache cassandra as a backend repository for data.
>> to do this, i would create a storage provider that handles the cassandra
>> interaction?

cheers
ashish

Permalink | Reply |

headers

Niklas Gustavsson | 18 Feb 15:47

Re: [vysper] questions

Niklas Gustavsson < niklas@...>
2011-02-18 14:47:14 GMT

On Fri, Feb 18, 2011 at 11:51 AM, Ashish <paliwalashish@...> wrote:
> Would like to hear more on what all we need to cluster :)
> I might be able to help out here a bit, as I am working on something similar.

I haven't really thought too much about this, so take this as me
thinking our loud.

XMPP uses DNS SRV to provide for fallback between multiple running
servers. But, as Bernd points out we would need to share the resource
registry among servers and also provide a way for servers to relay
messages in-between each other (user1@... on server1.vysper.org
sends a message to user2@... who happens to be on
server2.vysper.org).

I was kind of planning to suggest clustering support as a Google
Summer of Code project. Does that make sense?

/niklas