kubernetes权限RBAC之授权、鉴权、审计
文章目录
- 权限认证
- 一、kubernetes权限RBAC
- 二、验证权限的过程
- 三、代码分析
- 四、填充请求信息
- 五、多集群路由转发和协议升级
- 六、权限获取
- 七、k8s集群资源转发
- 总结
权限认证
一、kubernetes权限RBAC
kubernetes使用的RBAC的示意图如下：user1 可以使用role1和role2所允许的资源，user2只可以使用 role1 所允许的资源。我们有时候会称user为subject
在kubernetes，serviceaccount、user、group为user。、clusterrole和role代表role。clusterrolebinding和rolebinding代表rolebinding。
- GlobalRole（CRD资源）和ClusterRole、Role：全局角色，集群角色，项目角色，这三个都包含Rules，定义了这个Role所能允许的资源。三者的区别在于，在Role中是有namespace的，属于单个命名空间，ClusterRole是单个集群的，GlobalRole是多个集群的。
- Subject ：里面对应相应的用户如上图所示，Subject里面包含：apiGroup、Kind、Name（User的Name）
- RoleBinding、ClusterRoleBinding、GlobalRoleBinding：将Role与Subject绑定。确定每一个用户都有确定的Rules（权限）。
二、验证权限的过程
- 填充请求。确定请求的范围，单集群还是多集群，是否是发给k8s的请求
- 验证用户，密码、token等方式，如果是无用户则定义为anonymous
- 如果是多集群，则需要转发。
- 确定该资源是否能被用户获取
1. 通过user，globalRoleBinding获取到globalRole，查看GlobalRole对该资源是否允许。
2. 通过user，ClusterroleBinding获取到Clusterrole
3. 通过user，Rolebinding获取到Role，查看Role对资源是否允许
为了减少查询次数，通过if来过滤 - 审计
- 如果是k8s资源，类似于kubectl get pod -A的资源，则转发给kube-apiserver
三、代码分析
本项目使用的是iam模块来进行权限和权限获取，本项目使用go-restful开发 web应用框架，权限认证和权限获取都是在filter进行的
权限代码如下： **注意：filter和handle的方式一样，先被封装后被运行。**所以下面请求的执行过程是：
WithRequestInfo—>WithAuthentication—>WithMultipleClusterDispatcher—>WithAuthorization—>WithAuditing---->WithKubeAPIServer
func (s *APIServer) buildHandlerChain(stopCh <-chan struct{}) {
//.....
handler := s.Server.Handler
handler = filters.WithKubeAPIServer(handler, s.KubernetesClient.Config(), &errorResponder{})
//开启审计
if s.Config.AuditingOptions.Enable {
handler = filters.WithAuditing(handler,
audit.NewAuditing(s.InformerFactory, s.Config.AuditingOptions.WebhookUrl, stopCh))
}
//授权
handler = filters.WithAuthorization(handler, authorizers)
if s.Config.MultiClusterOptions.Enable {
//多集群转发
handler = filters.WithMultipleClusterDispatcher(handler, clusterDispatcher)
}
loginRecorder := im.NewLoginRecorder(s.KubernetesClient.KubeSphere())
//认证
handler = filters.WithAuthentication(handler, authn, loginRecorder)
//WithRequestInfo
handler = filters.WithRequestInfo(handler, requestInfoResolver)
s.Server.Handler = handler
}
主要分为以下步骤
- WithRequestInfo ：填充请求信息
- WithAuthentication：权限认证
- WithMultipleClusterDispatcher：多集群路径填补和转发，多集群时根据requestinfo，将信息发送给对应的集群。
- WithAuthorization：权限获取，判断是否对应集群资源有获取权限。
- WithAuditing：审计
- WithKubeAPIServer：转发，如果是直接对k8s的请求，则直接转发给k8s。
四、填充请求信息
请求信息如下：
type RequestInfo struct {
// import k8s.io/apiserver/pkg/endpoints/request
*k8srequest.RequestInfo
//是否是k8s的请求，如果是k8s的请求，则会直接转发到kube-apiserver
IsKubernetesRequest bool
//请求资源所在的Workspace，可以为空
Workspace string
//请求资源所在的Cluster，如果是单集群则为空
Cluster string
//请求资源所在的devops项目
DevOps string
// 资源请求范围
ResourceScope string
}
认证
有三个认证方式，anoymous、password、token三个方法任意一个通过校验，就立刻返回。
- anoymous的实现方法如下
func (a *Authenticator) AuthenticateRequest(req *http.Request) (*authenticator.Response, bool, error) {
auth := strings.TrimSpace(req.Header.Get("Authorization"))
if auth == "" {
return &authenticator.Response{
User: &user.DefaultInfo{
Name: user.Anonymous,
UID: "",
Groups: []string{user.AllUnauthenticated},
},
}, true, nil
}
return nil, false, nil
}
- password的实现方法如下：从user资源中获取信息
func (im *passwordAuthenticator) Authenticate(username, password string) (authuser.Info, error) {
//从k8s 集群中查询到user资源
user, err := im.searchUser(username)
providerOptions, ldapProvider := im.getLdapProvider()
//....
//通过ldapProvider来判断账号密码是否正确
if ldapProvider != nil && username != constants.AdminUserName {
//
authenticated, err := ldapProvider.Authenticate(username, password)
//....
if authenticated != nil {
return &authuser.DefaultInfo{
Name: authenticated.Name,
UID: string(authenticated.UID),
}, nil
}
}
//也可以通过use资源的EncryptedPassword，验证密码
if checkPasswordHash(password, user.Spec.EncryptedPassword) {
return &authuser.DefaultInfo{
Name: user.Name,
UID: string(user.UID),
}, nil
}
return nil, AuthFailedIncorrectPassword
}
- token：使用jwt验证
func (t tokenOperator) Verify(tokenStr string) (user.Info, error) {
//调用jwt包验证
authenticated, tokenType, err := t.issuer.Verify(tokenStr)
//这里应该是过期时间，0为永不过期。这个为配置设置
if t.options.OAuthOptions.AccessTokenMaxAge == 0 ||
tokenType == token.StaticToken {
return authenticated, nil
}
//redis验证
if err := t.tokenCacheValidate(authenticated.GetName(), tokenStr); err != nil {
klog.Error(err)
return nil, err
}
return authenticated, nil
}
五、多集群路由转发和协议升级
只在多集群下运行此函数
func (c *clusterDispatch) Dispatch(w http.ResponseWriter, req *http.Request, handler http.Handler) {
info, _ := request.RequestInfoFrom(req.Context())
//cluster 是crd 资源。获取集群所有cluster信息
cluster, err := c.clusterLister.Get(info.Cluster)
//请求集群是主机集群，不需要通过代理
if isClusterHostCluster(cluster) {
req.URL.Path = strings.Replace(req.URL.Path, fmt.Sprintf("/clusters/%s", info.Cluster), "", 1)
handler.ServeHTTP(w, req)
return
}
//查询集群
innCluster := c.getInnerCluster(cluster.Name)
transport := http.DefaultTransport
//替换url
u := *req.URL
u.Path = strings.Replace(u.Path, fmt.Sprintf("/clusters/%s", info.Cluster), "", 1)
if cluster.Spec.Connection.Type == clusterv1alpha1.ConnectionTypeDirect &&
len(cluster.Spec.Connection.KubeSphereAPIEndpoint) == 0 {
u.Scheme = innCluster.kubernetesURL.Scheme
u.Host = innCluster.kubernetesURL.Host
u.Path = fmt.Sprintf(proxyURLFormat, u.Path)
transport = innCluster.transport
//...
} else {
u.Host = innCluster.kubesphereURL.Host
u.Scheme = innCluster.kubesphereURL.Scheme
}
httpProxy := proxy.NewUpgradeAwareHandler(&u, transport, false, false, c)
httpProxy.ServeHTTP(w, req)
}
六、权限获取
首先在apiserver层中初始化Authorizer，然后进行授权模式的判断 代码如下：
var authorizers authorizer.Authorizer
switch a.Config.AuthorizationOptions.Mode{
case authorizationoptions.AlwaysAllow:
authorizers = authorizerfactory.NewAlwaysAllowAuthorizer()
case authorizationoptions.AlwaysDeny:
authorizers = authorizerfactory.NewAlwaysDenyAuthorizer()
default:
fallthrough
case authorizationoptions.RBAC:
//放行的所有路径：如登录等
excludePaths := []string{"/login/*"}
pathAuthorizer,_ :=path.NewAuthorizer(excludePaths)
//RBAC权限判断，url鉴权
RBACAuthorizer:=rbac.NewRBACAuthorizer(a.KubernetesClient.Kubernetes(),a.KubernetesClient.Kubesphere())
//将放行的路径和RBAC权限授权放到unionauthorizer里面，然后进行首先判断，是不是放行的路径，根据pathAuthorizer返回的Authorizer，来判断如果是放行的路径，直接放行，否则进行RBAC权限判定
authorizers = unionauthorizer.New(pathAuthorizer,RBACAuthorizer)
}
//将授权加到handler里面
handler = middleware.WithAuthorization(handler,authorizers)
path放行路径的截取
代码如下：
var prefixes []string
paths := sets.NewString()
for _, p := range alwaysAllowPaths {
//将放行的URL的前缀/去掉，如：/login/*去掉后为login/*
p = strings.TrimPrefix(p, "/")
if len(p) == 0 {
paths.Insert(p)
continue
}
//判断放行的URL是不是写的为/**
if strings.ContainsRune(p[:len(p)-1], '*') {
return nil, fmt.Errorf("only trailing * allowed in %q", p)
}
//去掉后面的*
if strings.HasSuffix(p, "*") {
prefixes = append(prefixes, p[:len(p)-1])
} else {
paths.Insert(p)
}
}
return authorizer.AuthorizerFunc(func(a authorizer.Attributes) (authorizer.Decision, string, error) {
pth := strings.TrimPrefix(a.GetPath(), "/")
//如果是不带*的则需要用户请求的路径和放行的路径完全相同
if paths.Has(pth) {
return authorizer.DecisionAllow, "", nil
}
//如果是/*的则只要前缀为例如/login的即可
for _, prefix := range prefixes {
if strings.HasPrefix(pth, prefix) {
//放行
return authorizer.DecisionAllow, "", nil
}
}
return authorizer.DecisionNoOpinion, "", nil
}), nil
Union来判断是放行的路径，还是进行RBAC授权
//...authorizer.Authorizer 代表可以存放多个类型为Authorizer的结构体（即：放行的路径、RBAC授权）
func New(authorizationHandlers ...authorizer.Authorizer) authorizer.Authorizer {
return unionAuthzHandler(authorizationHandlers)
}
func (authzHandler unionAuthzHandler) Authorize(a authorizer.Attributes) (authorizer.Decision, string, error) {
var (
errlist []error
reasonlist []string
)
for _, currAuthzHandler := range authzHandler {
decision, reason, err := currAuthzHandler.Authorize(a)
if err != nil {
errlist = append(errlist, err)
}
if len(reason) != 0 {
reasonlist = append(reasonlist, reason)
}
//根据截取放行的路径返回的放行DecisionAllow，进行放行
switch decision {
case authorizer.DecisionAllow, authorizer.DecisionDeny:
return decision, reason, err
case authorizer.DecisionNoOpinion:
// continue to the next authorizer
}
}
return authorizer.DecisionNoOpinion, strings.Join(reasonlist, "\n"), utilerrors.NewAggregate(errlist)
}
RBAC用户授权
func appliesTo(user user.Info, bindingSubjects []rbacv1.Subject, namespace string) (int, bool) {
for i, bindingSubject := range bindingSubjects {
if appliesToUser(user, bindingSubject, namespace) {
return i, true
}
}
return 0, false
}
//根据请求的用户去查看是否有该用户
func appliesToUser(user user.Info, subject rbacv1.Subject, namespace string) bool {
switch subject.Kind {
case rbacv1.UserKind:
a :=strings.Compare(user.GetName(),subject.Name)
return a == 0
default:
return false
}
}
RBAC权限匹配
func ruleAllows(requestAttributes authorizer.Attributes, rule *rbacv1.PolicyRule) bool {
//判断是否是资源请求
if requestAttributes.IsResourceRequest() {
//如果含有子资源的请求，则进行拼接
combinedResource := requestAttributes.GetResource()
if len(requestAttributes.GetSubresource()) > 0 {
combinedResource = requestAttributes.GetResource() + "/" + requestAttributes.GetSubresource()
}
//进行权限、APIGroup、资源、资源名字进行判断用户请求的URL和该用户拥有的权限是否一致
return VerbMatches(rule, requestAttributes.GetVerb()) &&
APIGroupMatches(rule, requestAttributes.GetAPIGroup()) &&
ResourceMatches(rule, combinedResource, requestAttributes.GetSubresource()) &&
ResourceNameMatches(rule, requestAttributes.GetName())
}
return VerbMatches(rule, requestAttributes.GetVerb())
}
权限获取是通过RBAC方式。先调用Authorize
func (r *RBACAuthorizer) Authorize(requestAttributes authorizer.Attributes) (authorizer.Decision, string, error) {
//调用visitRulesFor检查权限
r.visitRulesFor(requestAttributes, ruleCheckingVisitor.visit)
if ruleCheckingVisitor.allowed {
return authorizer.DecisionAllow, ruleCheckingVisitor.reason, nil
}
//...
//记录日志
klog.Infof("...")
//...
return authorizer.DecisionNoOpinion, reason, nil
}
在调用vistRulesFor
func (r *RBACAuthorizer) visitRulesFor(requestAttributes authorizer.Attributes, visitor func(source fmt.Stringer, regoPolicy string, rule *rbacv1.PolicyRule, err error) bool) {
//查看globalRole是否有对此资源的获取权限
if globalRoleBindings, err := r.am.ListGlobalRoleBindings(""); err != nil {
//...
} else {
sourceDescriber := &globalRoleBindingDescriber{}
//循环globalRoleBindings。找到该用户的globalRoleBindings。
for _, globalRoleBinding := range globalRoleBindings {
subjectIndex, applies := appliesTo(requestAttributes.GetUser(), globalRoleBinding.Subjects, "")
if !applies {
//如果不是这个找到该用户的globalRoleBindings，则continue
continue
}
//根据globalRoleBinding上的roleref，获取到role。进一步获取到regoPolicy和rules。这是两个规则，只要有一个规则符合即可
regoPolicy, rules, err := r.am.GetRoleReferenceRules(globalRoleBinding.RoleRef, "")
//...
//根据regoPolicy验证是否符合规则
if !visitor(sourceDescriber, regoPolicy, nil, nil) {
return
}
//根据rules验证是否符合规则
for i := range rules {
if !visitor(sourceDescriber, "", &rules[i], nil) {
return
}
}
}
//...
}
if requestAttributes.GetResourceScope() == request.WorkspaceScope ||
requestAttributes.GetResourceScope() == request.NamespaceScope ||
requestAttributes.GetResourceScope() == request.DevOpsScope {
//...
// 代码部分删除，这部分主要是获取workspace
workspace, err = r.am.GetNamespaceControlledWorkspace(requestAttributes.GetNamespace()); err != nil
// 通过workspace获取workspaceRoleBindings。
if workspaceRoleBindings, err := r.am.ListWorkspaceRoleBindings("", workspace); ...{
//...
} else {
//...轮训workspaceRoleBindings，找到workspaceRole，在进行权限检测。这部分和grobalrole一样
for _, workspaceRoleBinding := range workspaceRoleBindings {
subjectIndex, applies := appliesTo(requestAttributes.GetUser(), workspaceRoleBinding.Subjects, "")
if !applies {
continue
}
regoPolicy, rules, err := r.am.GetRoleReferenceRules(workspaceRoleBinding.RoleRef, "")
//...
if !visitor(sourceDescriber, regoPolicy, nil, nil) {
return
}
for i := range rules {
if !visitor(sourceDescriber, "", &rules[i], nil) {
return
}
}
}
}
}
if requestAttributes.GetResourceScope() == request.NamespaceScope ||
requestAttributes.GetResourceScope() == request.DevOpsScope {
namespace := requestAttributes.GetNamespace()
// 直接获取namespace，或者根据DevOps获取namespace
if requestAttributes.GetResourceScope() == request.DevOpsScope {
if relatedNamespace, err := r.am.GetDevOpsRelatedNamespace(requestAttributes.GetDevOps()); err != nil {
if !visitor(nil, "", nil, err) {
return
}
} else {
namespace = relatedNamespace
}
}
//根据namespace获取rolebinding
if roleBindings, err := r.am.ListRoleBindings("", namespace); err != nil {
if !visitor(nil, "", nil, err) {
return
}
} else {
sourceDescriber := &roleBindingDescriber{}
//轮训roleBindings，找到role,检查role
for _, roleBinding := range roleBindings {
subjectIndex, applies := appliesTo(requestAttributes.GetUser(), roleBinding.Subjects, namespace)
if !applies {
continue
}
regoPolicy, rules, err := r.am.GetRoleReferenceRules(roleBinding.RoleRef, namespace)
if err != nil {
visitor(nil, "", nil, err)
continue
}
sourceDescriber.binding = roleBinding
sourceDescriber.subject = &roleBinding.Subjects[subjectIndex]
if !visitor(sourceDescriber, regoPolicy, nil, nil) {
return
}
for i := range rules {
if !visitor(sourceDescriber, "", &rules[i], nil) {
return
}
}
}
}
}
//获取所有clusterRoleBindings，轮训查找到clusterRole。检查规则
if clusterRoleBindings, err := r.am.ListClusterRoleBindings(""); err != nil {
if !visitor(nil, "", nil, err) {
return
}
} else {
sourceDescriber := &clusterRoleBindingDescriber{}
for _, clusterRoleBinding := range clusterRoleBindings {
subjectIndex, applies := appliesTo(requestAttributes.GetUser(), clusterRoleBinding.Subjects, "")
if !applies {
continue
}
regoPolicy, rules, err := r.am.GetRoleReferenceRules(clusterRoleBinding.RoleRef, "")
if err != nil {
visitor(nil, "", nil, err)
continue
}
sourceDescriber.binding = clusterRoleBinding
sourceDescriber.subject = &clusterRoleBinding.Subjects[subjectIndex]
if !visitor(sourceDescriber, regoPolicy, nil, nil) {
return
}
for i := range rules {
if !visitor(sourceDescriber, "", &rules[i], nil) {
return
}
}
}
}
}
检查规则的函数是visitor，如果regoPolicy符合就不检查rule
func (v *authorizingVisitor) visit(source fmt.Stringer, regoPolicy string, rule *rbacv1.PolicyRule, err error) bool {
//调用open-policy-agent库检查权限
if regoPolicy != "" && regoPolicyAllows(v.requestAttributes, regoPolicy) {
v.allowed = true
v.reason = fmt.Sprintf("RBAC: allowed by %s", source.String())
return false
}
//调用k8s 接口实现rbac检查权限
if rule != nil && ruleAllows(v.requestAttributes, rule) {
v.allowed = true
v.reason = fmt.Sprintf("RBAC: allowed by %s", source.String())
return false
}
if err != nil {
v.errors = append(v.errors, err)
}
return true
}
七、k8s集群资源转发
如果这个资源是向k8s请求的，则直接向k8s请求
func WithKubeAPIServer(handler http.Handler, config *rest.Config, failed proxy.ErrorResponder) http.Handler {
//...
//这部分为初始化内容，在请求时不执行。
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
info, ok := request.RequestInfoFrom(req.Context())
if !ok {
err := errors.New("Unable to retrieve request info from request")
klog.Error(err)
responsewriters.InternalError(w, req, err)
}
// 确认是否为k8s请求
if info.IsKubernetesRequest {
s := *req.URL
s.Host = kubernetes.Host
s.Scheme = kubernetes.Scheme
// make sure we don't override kubernetes's authorization
req.Header.Del("Authorization")
//转发
httpProxy := proxy.NewUpgradeAwareHandler(&s, defaultTransport, true, false, failed)
httpProxy.UpgradeTransport = proxy.NewUpgradeRequestRoundTripper(defaultTransport, defaultTransport)
httpProxy.ServeHTTP(w, req)
return
}
handler.ServeHTTP(w, req)
})
}
总结
希望与大佬们进行交流学习 。如有侵权部分请联系删除侵权内容